Motion vector calculation method

ABSTRACT

When a block (MB 22 ) of which motion vector is referred to in the direct mode contains a plurality of motion vectors, 2 motion vectors MV 23  and MV 24 , which are used for inter picture prediction of a current picture (P 23 ) to be coded, are determined by scaling a value obtained from averaging the plurality of motion vectors or selecting one of the plurality of the motion vectors.

TECHNICAL FIELD

The present invention relates to a moving picture coding method and adecoding method, and particularly to a prediction coding methodreferring to plural coded pictures preceding in display order or pluralcoded pictures following in display order or plural pictures bothpreceding and following in display order.

BACKGROUND ART

Generally, information volume is compressed by reducing redundancy intemporal and spatial directions for moving picture coding. Therefore,motion deriving and motion compensation are performed on ablock-to-block basis referring to a preceding or a following picture,and a coding is performed for a difference value between an obtainedpredictive picture and a current picture for inter picture predictioncoding aimed at reducing a temporal redundancy.

In a moving picture coding method H.26L, which is currently understandardization, a picture with only intra picture prediction coding (Ipicture), a picture for which inter picture prediction coding isperformed referring to one picture (hereinafter P picture) and a picturefor which inter picture prediction coding is performed referring to twopictures preceding in display order or two pictures following in displayorder or each one of pictures preceding and following in display order(hereinafter B picture) are proposed.

FIG. 1 is an illustration showing an example of a reference relationbetween each picture according to above-mentioned moving picture codingmethod and reference pictures.

In picture I1 intra picture prediction coding is performed without areference picture, and in picture P10 inter picture prediction coding isperformed referring to a picture preceding in display order, P7. In apicture B6 inter picture prediction coding is performed referring to twopictures preceding in display order, in a picture B12 inter pictureprediction coding is performed referring to two pictures following indisplay order and in a picture B18 inter picture prediction coding isperformed referring to each one of pictures preceding and following indisplay order.

A direct mode is one of prediction mode of bi-predictions which performinter picture prediction coding referring to each of pictures precedingand following in display order. In the direct mode, motion vectors for ablock to be coded are not coded in the bit stream directly, and twomotion vectors for actual motion compensation are calculated referringto a motion vector of a co-located block in a coded picture close to thepicture including the block to be coded in display order, and apredictive block is generated.

FIG. 2 shows an example that a coded picture which is referred to inorder to determine a motion vector in the direct mode contains a motionvector which refers to a preceding picture in display order. “P”indicated by a vertical line in FIG. 2 has nothing to do with a picturetype and it shows a mere picture. In FIG. 2, for example, a picture P83,in which bi-prediction is performed referring to pictures P82 and P84,is a current picture to be coded. If it is assumed that a block withcoding in the picture P83 is a block MB81, a motion vector of the blockMB81 is determined using a motion vector of a co-located block MB82 inthe picture P84 which is a coded backward reference picture. Since theblock MB82 contains only one motion vector MV81 as a motion vector, twomotion vectors MV82 and MV83 to be obtained are calculated directly byapplying a scaling to a motion vector MV81 and a time interval TR81based on Equation 1 (a) and Equation 1 (b).MV82=MV81/TR81×TR82  Equation 1 (a)MV83=−MV81/TR81×TR83  Equation 1 (b)

In these equations, the time interval TR81 shows an interval between thepicture P84 and the picture P82, that is, a time interval between thepicture P84 and a reference picture indicated by the motion vector MV81.The time interval TR82 shows a time interval between the picture P83 anda reference picture indicated by the motion vector MV82. The timeinterval TR83 shows a time interval between the picture P83 and areference picture indicated by the motion vector MV83.

The direct mode includes two methods, the temporal prediction alreadyexplained and the spatial prediction, and the spatial prediction isexplained below. In the spatial prediction in the direct mode, forexample, coding is performed on a macroblock of 16×16 pixels basis, anda motion vector, which is obtained referring to a picture closest from acurrent picture to be coded in display order, is selected from motionvectors in three macroblocks neighboring the current macroblock to becoded, and the selected motion vector is a motion vector for the currentmacroblock to be coded. If three motion vectors refer to a same picture,a median value is selected. If two of three motion vectors refer to apicture closest from a current picture to be coded in display order, theremainder is considered as “0” vector, and a median value of thesevalues is selected. If only 1 motion vector refers to a picture closestfrom a current picture to be coded in display order, this motion vectoris selected. Thus a motion vector is not coded for a current macroblockto be coded in the direct mode, and motion prediction is performed usinga motion vector contained in another macroblock.

FIG. 3A is an illustration showing an example of a motion vectorpredicting method in the case that a picture preceding in a B picture indisplay order is referred to using a conventional spatial predictingmethod in the direct mode. In this FIG. 3A, P indicates a P picture, Bindicates a B picture and numbers assigned to picture types in rightfour pictures indicate an order in which each picture is coded. Itshould be assumed that a macroblock diagonally shaded in a picture B4 isa current macroblock to be coded. When a motion vector of a currentmacroblock to be coded is calculated using a spatial predicting methodin the direct mode, first, three coded macroblocks (area shaded withbroken lines) are selected from macroblocks neighboring the currentmacroblock to be coded. Explanation of a method for selecting threeneighboring macroblocks is omitted here. Motion vectors in coded threemacroblocks have been calculated and stored already. There is a casethat the motion vector is obtained referring to different pictures foreach macroblock even if macroblocks are in a same picture. Referenceindices in reference pictures used for coding each macroblock can showwhich picture is referred to by the three neighboring macroblocksrespectively from. Detail of reference indices will be explained later.

Now, for example, it is assumed that three neighboring macroblocks areselected for a current macroblock to be coded shown in FIG. 3A, andmotion vectors in each coded macroblock are a motion vector a, b and crespectively. Here, it is assumed that the motion vector and the motionvector b are obtained referring to a P picture with a picture number 11of “11”, and the motion vector c is obtained referring to a P picturewith the picture number 11 of “8”. In this case, among these motionvectors, a, b, and c, the motion vectors a and b which refer to apicture closest to a current picture to be coded in order or displaytime are candidates for a motion vector of a current macroblock to becoded. In this case, the motion vector c is considered as “0”, and amedian value of these three motion vectors a, b and c is selected anddetermined as a motion vector of the current macroblock to be coded.

However, a coding method such as MPEG-4 can perform coding for eachmacroblock in a picture using a field structure and a frame structure.Therefore, in a coding method such as MPEG-4, there is a case that amacroblock coded in the field structure and a macroblock coded in theframe structure are mixed in one frame of reference frame. Even in sucha case, if three macroblocks neighboring a current macroblock to becoded are coded in the same structure as the current macroblock to becoded, it is possible to derive a motion vector of the currentmacroblock to be coded using the above-mentioned spatial predictingmethod in the direct mode without any problems. That is, a case thatthree neighboring macroblocks are coded in the frame structure for acurrent macroblock to be coded in the frame structure, or a case thatthree neighboring macroblocks are coded in the field structure for acurrent macroblock to be coded in the field structure. The former caseis as already explained. In the latter case, by using three motionvectors corresponding to top fields of three neighboring macroblocks fora top field of a current macroblock to be coded, and by using threemotion vectors corresponding to bottom fields of three neighboringmacroblocks for a bottom field of the current macroblock to be coded, amotion vector of the current macroblock to be coded can be derived forthe top field and the bottom field respectively using theabove-mentioned method.

However, in the temporal prediction method in the direct mode, since theabove-mentioned block contains plural motion vectors for temporalprediction in the direct mode when in a block with intra pictureprediction coding, motion compensation in the direct mode is performed,if a block of which motion vector is referred to belongs to a B picturesuch as B6 shown in FIG. 1, a problem occurs because a calculation ofmotion vector by a scaling based on Equation 1 can not be applieddirectly. Furthermore, there is a case that precision of motion vectorvalue (half pixel precision and quarter pixel precision, for example)does not meet predetermined precision since dividing operation isperformed after the calculation of motion vector.

When a current macroblock to be coded and one of neighboring macroblocksare coded in a different structure for a spatial prediction, it is notspecified which one of a field structure or a frame structure is usedfor coding the current macroblock to be coded, and a method forselecting a motion vector of the current macroblock to be coded frommotion vectors of neighboring macroblocks coded in both the fieldstructure and the frame structure is not specified.

The first object of the present invention is to offer a motion vectorprediction method in temporal direction with high precision in thedirect mode even if a block of which motion vector is referred tobelongs to a B picture.

The second object of the present invention is to offer a motion vectorprediction method in spatial direction with high precision in the directmode even if a block of which motion vector is referred to belongs to aB picture.

SUMMARY OF THE INVENTION

In order to achieve above object, a motion vector calculation methodaccording to the present invention is a motion vector calculation methodfor inter picture prediction with reference to a plurality of pictures.The motion vector calculation method comprises a referring step ofreferring to a plurality of pictures preceding in display order or aplurality of pictures following in display order or a plurality ofpictures both preceding and following in display order, and a motioncompensating step, with reference to a motion vector of a co-locatedblock in a picture other than a picture to which an inter picturepredictive block belongs when performing motion compensation of theinter picture predictive block, for calculating a motion vector of theinter picture predictive block using at least one motion vector, whichsatisfies a predetermined condition, among motion vectors alreadycalculated for the co-located block of which motion vector is referredto. Therefore according to the motion vector calculation method of thepresent invention, when in a block with inter picture predictive codingmotion compensation is performed referring to a motion vector of aco-located block in a coded other picture, and when a block of whichmotion vector is referred to contains a plurality of motion vectors, itis possible to actualize the motion compensation without contradictionby generating one motion vector used for scaling among the plurality ofmotion vectors.

For the motion vector calculation method according to the presentinvention, in the reference step, each one of pictures selected from afirst picture order and a second picture order can be referred to. Herethe first picture order is the order in which identifying numbers areassigned to pictures in ascending sequence giving priority to a pictureprecedent in display order, and the second picture order is the order inwhich identifying numbers are assigned to pictures in ascending sequencegiving priority to a picture following in display order. In the motioncompensating step, a motion vector, which refers to a picture in thefirst picture order, of the block of which motion vector is referred tomay be used. For the method, even if a block of which motion vector isreferred to belongs to the B picture, the one used for the motioncompensation of the block with inter picture prediction can bedetermined as a motion vector referring to a picture in the firstpicture order, and motion vector calculation method by scaling can beapplied.

Moreover, another motion vector calculation method according to thepresent invention includes an assigning step, a first selecting step anda deriving step. The assigning step is for assigning one of a firstreference index or a second reference index to the coded picture. Here,the first reference index and the second reference index are used forselecting at least one of a first reference picture or a secondreference picture which are referred to when obtaining a block in acurrent picture to be coded by motion compensation from a plurality ofcoded pictures stored in a storing unit. The first selecting step is forselecting a motion vector indicating a median value of the motionvectors, when performing motion compensation of a block in the currentpicture to be coded, there are a plurality of motion vectors containingthe first reference indices among motion vectors of blocks in theneighbor of the current picture to be coded. The deriving step is forderiving a motion vector referring to a picture preceding the currentpicture to be coded in display order or a picture following the currentpicture to be coded in display order or pictures preceding and followingthe current picture to be coded in display order using the motion vectorselected in the first selecting step. Therefore, when motioncompensation is performed for a block in the current picture to be codedand when there are a plurality of motion vectors containing the firstreference index among motion vectors of blocks in the neighbor of theblock in the current picture to be coded, the motion vector of thecurrent picture to be coded can be derived using a motion vectorindicating a median value of motion vectors.

For the motion vector calculation method according to the presentinvention, a motion vector indicating a median value of the smallestfirst reference indices may be further selected from motion vectorscontaining the first reference indices in the first selecting step.

Further Information about Technical Background to this Application

Japanese Patent Application No. 2002-118598 filed Apr. 19, 2002:

Japanese Patent Application No. 2002-121053 filed Apr. 23, 2002:Japanese Patent Application No. 2002-156266 filed May 29, 2002:

Japanese Patent Application No. 2002-177889 filed Jun. 19, 2002:

Japanese Patent Application No. 2002-193027 filed Jul. 2, 2002:

Japanese Patent Application No. 2002-204713 filed Jul. 12, 2002:

Japanese Patent Application No. 2002-262151 filed Sep. 6, 2002:

Japanese Patent Application No. 2002-290542 filed Oct. 2, 2002:

Japanese Patent Application No. 2002-323096 filed Nov. 6, 2002:

U.S. Provisional Application No. 60/378,643 filed May 9, 2002:

U.S. Provisional Application No. 60/378,954 filed May 10, 2002:

are incorporated herein by reference.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram showing a referential relation of picturesof a conventional example.

FIG. 2 is a schematic diagram showing an operation in a conventionaldirect mode.

FIG. 3A is an illustration showing an example of a motion vectorpredicting method when a temporally preceding picture is referred to ina B picture using a spatial predicting method of a conventional directmode.

FIG. 3B is an illustration showing an example of a reference listgenerated in each current picture to be coded.

FIGS. 4A and 4B are explanatory illustrations of picture numbers andreference indices.

FIG. 5 is an illustration showing a concept of a picture coding signalformat of a conventional picture coding apparatus.

FIG. 6 is a block diagram showing an operation of coding according tothe first and the second embodiments of this invention.

FIG. 7 is a schematic diagram showing an operation when a block of whichmotion vector is referred to in the direct mode contains two motionvectors which refer to preceding time in display order.

FIGS. 8A and 8B are schematic diagrams comparing a referential relationof pictures in display order and coding order.

FIG. 9 is a schematic diagram showing an operation when a block of whichmotion vector is referred to in the direct mode contains two motionvectors which refer to following time in display order.

FIGS. 10A and 10B are schematic diagrams comparing a referentialrelation of pictures in the display order and the coding order.

FIG. 11 is a block diagram showing an operation of decoding according tothe fifth and sixth embodiments of the present invention.

FIG. 12 is a schematic diagram showing an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to preceding time in display order.

FIG. 13 is a schematic diagram to show an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to following time in display order.

FIG. 14 is a schematic diagram to show an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to following time in display order.

FIG. 15 is a schematic diagram to show an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to preceding time in display order.

FIG. 16 is a schematic diagram to show an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to preceding time in display order.

FIG. 17 is a schematic diagram to show an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to preceding time in display order.

FIG. 18 is a schematic diagram to show an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to preceding time in display order.

FIG. 19 is a schematic diagram to show an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to following time in display order.

FIG. 20 is a schematic diagram to show an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to following time in display order.

FIG. 21 is a schematic diagram to show an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to following time in display order.

FIG. 22 is a schematic diagram to show an operation when a block ofwhich motion vector is referred to in the direct mode contains twomotion vectors which refer to following time in display order.

FIG. 23 is a schematic diagram to show an operation when a motion vectorof a neighboring block is referred to in the direct mode.

FIGS. 24A and 24B are illustrations showing a bit stream.

FIG. 25 is an illustration showing a relation between a current block tobe coded and a block in the neighbor of the current block to be coded.

FIG. 26 is an illustration showing a motion vector contained in a blockneighboring a current block to be coded.

FIG. 27 is an illustration showing a motion vector contained in a blockneighboring a current block to be coded.

FIG. 28 is an illustration showing a motion vector contained in a blockneighboring a current block to be coded.

FIG. 29 is an illustration showing a motion vector contained in a blockneighboring a current block to be coded.

FIG. 30 is an illustration showing a motion vector contained in a blockneighboring a current block to be coded.

FIG. 31 is an illustration showing a motion vector contained in a blockneighboring a current block to be coded.

FIG. 32 is an illustration showing a motion vector contained in a blockneighboring a current block to be coded.

FIG. 33 is an illustration showing a motion vector contained in a blockneighboring a current block to be coded.

FIGS. 34A and 34B are illustrations showing a procedure for determininga motion vector to be used in the direct mode.

FIGS. 35A and 35B are illustrations showing a relation between a currentblock to be coded and a block in the neighbor of the current block to becoded.

FIGS. 36A, 36B, 36C and 36D are illustrations showing a procedure fordetermining a motion vector of a current block to be coded using a valueof a reference index.

FIG. 37 is an illustration showing bi-prediction in the direct mode whena motion vector referring to a picture stored in a long term picturebuffer is only one.

FIG. 38 is an illustration showing bi-prediction in the direct mode whenmotion vectors referring to a picture stored in the long term picturebuffer are two.

FIG. 39 is an illustration showing a process flow of a motion vectorcalculation method.

FIG. 40 is a block diagram showing a configuration of a moving picturecoding apparatus 100 according to the eleventh embodiment of the presentinvention.

FIG. 41A is an illustration showing an order of frames inputted into themoving picture coding apparatus 100 in order of time onpicture-to-picture basis.

FIG. 41B is an illustration showing the case that the order of framesshown in FIG. 41A is reordered in the coding order.

FIG. 42 is an illustration showing a structure of a reference picturelist to explain the first embodiment.

FIG. 43A is a flow chart showing an example of a motion vectorcalculation procedure using a spatial predicting method in the directmode when a macroblock pair to be coded in a field structure and amacroblock pair to be coded in a frame structure are mixed.

FIG. 43B is an illustration showing an example of a location ofneighboring macroblock pairs to which the present invention is appliedwhen a current macroblock pair to be coded is coded in a framestructure.

FIG. 43C is an illustration showing an example of location ofneighboring macroblock pairs to which the present invention is appliedwhen a current macroblock pair to be coded is coded in a fieldstructure.

FIG. 44 is an illustration showing a data configuration of a macroblockpair when coding is performed in a frame structure, and a dataconfiguration of a macroblock pair when coding is performed in a fieldstructure.

FIG. 45 is a flow chart showing a detailed processing procedure in astep S302 shown in FIG. 43.

FIG. 46 is an indicator chart showing a relation between reference fieldindices and reference frame indices.

FIG. 47 is a flow chart showing a detailed processing procedure in astep S303 shown in FIG. 43.

FIGS. 48A and 48B are illustrations showing a relation of positionbetween a current macroblock pair to be coded and neighboring macroblockpairs in order to explain the first embodiment.

FIGS. 49A and 49B are illustrations showing a positional relationbetween a current macroblock pair to be coded and neighboring macroblockpairs in order to explain the first embodiment.

FIG. 50 is an illustration showing an example of a data configuration ofa bit stream 700 generated by a bit stream generating unit 104.

FIG. 51 is a block diagram showing a configuration of a moving picturedecoding apparatus 800 which decodes the bit stream 700 shown in FIG.50.

FIG. 52A is an illustration showing an example of a physical format of aflexible disk which is a body of a storage medium.

FIG. 52B is an illustration showing an external view of the flexibledisk viewed from the front, a configuration of the section and theflexible disk.

FIG. 52C is an illustration showing a configuration to record and readthe above-mentioned program on a flexible disk, FD.

FIG. 53 is a block diagram showing an entire configuration of contentssupply system implementing a contents delivery service.

FIG. 54 is an illustration showing an example of an appearance of a cellphone.

FIG. 55 is a block diagram showing a configuration of the cell phone.

FIG. 56 is an illustration to show a device performing the coding or thedecoding process shown in above embodiments and a system using thedevice.

DETAILED DESCRIPTION OF THE INVENTION

The present invention aims to solve problems of the conventionaltechnology, and aims at proposing a moving picture coding method and adecoding method which can determine a motion vector used for motioncompensation without contradiction even if a block of which motionvector is referred to in a direct mode is a B picture. First, referenceindices are explained here.

FIG. 3B is an illustration showing an example of a reference picturelist 10 generated for each current picture to be coded. In the referencepicture list 10 shown in FIG. 3B, pictures are shown preceding andfollowing a B picture in display order with one B picture at the center,and pictures to which the B picture can refer, picture types, a picturenumber 11, the first reference index 12 and the second reference index13 are shown. The picture number 11 is, for example, a number showing anorder in which each picture is coded. The first reference index 12 isthe first index showing a relative positional relation between a currentpicture to be coded and neighboring pictures, and, for example, is usedmainly as an index when a current picture to be coded refers to apicture preceding in display order. A list of the first reference index12 is called a “reference index list0 (list0)” or “the first referenceindex list”. Moreover, the reference index is called a relative index.First, in the reference picture list 10 shown in FIG. 3B, integer whichis advanced by “1” is assigned to a value of the first reference index12 from “0” from the closest to a current picture to be coded in a timesequence for a reference picture preceding a current picture to be codedin display order. Next, after a value advanced by “1” from “0” isassigned to all reference pictures following a current picture to becoded in display order, following values are assigned to referencepictures following the current picture to be coded in display order fromclosest to the current picture to be coded in display order.

The second reference index 13 is the second index showing a relativepositional relation between a current picture to be coded andneighboring pictures, and, for example, is used mainly as an index whena current picture to be coded refers to a picture following in displayorder. A list of the second reference index 13 is called “referenceindex list1 (list1)” or “the second reference index list”. First,integer which is advanced by “1” is assigned to a value of the secondreference index 13 is from “0” from the closest to a current picture tobe coded in display order. Next, after a value advanced by “1” from “0”is assigned to all reference pictures following a current picture to becoded in display order, following values are assigned to referencepictures preceding a current picture to be coded in display order fromthe closest value to a current picture to be coded in display order.Therefore, it is found in the reference picture list 10 that as for thefirst reference index 12 and the second reference index, a referencepicture with smaller reference index value is closer to the currentpicture to be coded in display order. A method for assigning a referenceindex number in initial state of is explained above, however, the methodfor assigning a reference index number can be changed on apicture-by-picture basis or a slice-by-slice basis. In the method forassigning a reference index number, for example, a small number may beassigned to a picture far in display order, however, such a referenceindex is used, for example, when coding efficiency is improved byreferring to the picture far in display order. In other words, sincereference indices in a block are presented by variable length code wordsand data with shorter lengths are assigned to the indices of the smallervalues, by assigning smaller reference index to the reference picturewhich improves coding efficiency if it is referred to, the amount ofcodes in reference indices is reduced and further coding efficiency isimproved.

FIGS. 4A and 4B are explanatory illustrations for picture numbers andreference indices. FIGS. 4A and 4B show examples of the referencepicture list, and show a reference picture, a picture number and areference index used when coding the B picture at the center (indicatedby a broken line). FIG. 4A shows the case assigning reference indices bythe method for assigning reference indices in initial state explainedusing FIG. 3.

FIG. 5 is a conceptual diagram of a picture coding signal format of aconventional picture coding apparatus. Picture indicates a coding signalfor one picture, Header indicates a header coding signal included in thehead of a picture, Block1 indicates a coding signal in a block coded ina direct mode, Block2 indicates a coding signal in a block coded by aninterpolation prediction other than the direct mode, Ridx0 and Ridx1 arethe first reference index and the second reference index respectively,and MV0 and MV1 are the first motion vector and the second motion vectorrespectively. The coded block Block2 has two reference indices Ridx0 andRidx1 in a coding signal in this order for indicating two referencepictures to be used for motion compensation. Moreover, the first motionvector MV0 and the second motion vector MV1 are coded in the codingsignal of the coded block Block2 in this order. It can be judged byPredType that which of the reference indices Ridx0 and/or Ridx1 is used.A picture (the first reference picture) referred to by the first motionvector MV0 is indicated by the first reference index Ridx0, and apicture (the second reference picture) referred to by the second motionvector MV1 is indicated by the second reference index Ridx1. Forexample, when it is indicated that pictures are referred tobi-directionally by the motion vectors MV0 and MV1, Ridx0 and Ridx1 areused, when it is indicated that pictures are referred touni-directionally by one of motion vector MV0 or MV1, one of Ridx0 orRidx1 corresponding to the motion vector is used, and when the directmode is indicated, neither Ridx0 nor Ridx1 are used. The first referencepicture is specified by the first reference index and generally hasdisplay time preceding a current picture to be coded, and the secondreference picture is specified by the second reference index andgenerally has display time following the current picture to be coded.However, as the method for assigning reference indices in FIG. 4 shows,there is a case that the first reference picture contains display timefollowing the current picture to be coded and the second referencepicture contains display time preceding the current picture to be coded.The first reference index Ridx0 is a reference index indicating thefirst reference picture referred to by the first motion vector MV0 ofthe block Block2, and the second reference index Ridx1 is a referenceindex indicating the second reference picture referred to by the secondmotion vector MV1 of the block Block2.

On the other hand, an assignment of reference pictures to referenceindices can be changed arbitrarily by indicating explicitly using amemory control signal in a coded signal (RPSL in Header in FIG. 5). Thismakes it possible to change the reference picture with the secondreference index “0” to an arbitrary reference picture. For example, asshown in FIG. 4B, assignment of reference indices to picture numbers canbe changed.

Thus, since assignment of reference pictures to reference indices can bechanged arbitrarily and the change of the assignment of referencepictures to reference indices generally assigns a smaller referenceindex to a picture which improves coding efficiency if selected as areference picture, coding efficiency can be improved by using a motionvector, which refers to a picture of which reference index is thesmallest, as a motion vector used in the direct mode.

First Embodiment

A moving picture coding method according to the first embodiment of thepresent invention is explained using the block diagram shown in FIG. 6.

A current moving picture to be coded is inputted into a frame memory 101in a display order on a picture-to-picture basis, and reordered in acoding order. Each picture is divided into a group called a block, whichis 16 (horizontal)×16 (vertical) pixels in size, for example, andfollowing processes are performed on a block-to-block basis.

A block read from the frame memory 101 is inputted into a motion vectordetecting unit 106. Here, a motion vector of a current block to be codedis detected using a decoded picture of a coded picture stored in theframe memory 105 as a reference picture. In this case, in a modeselecting unit 107, an optimum prediction mode is determined withreference to a motion vector obtained in the motion vector detectingunit 106 and a motion vector used in a coded picture stored in a motionvector storing unit 108. A prediction mode obtained in the modeselecting unit 107 and a motion vector used in the obtained mode areinputted to a difference calculating unit 109, and a predictive residualpicture is generated by calculating a difference from a current block tobe coded, and coding is performed in a predictive residual coding unit102. Moreover, the motion vector used in the mode obtained in the modeselecting unit 107 is stored in a motion vector storing unit 108 inorder to be used for coding by following blocks and pictures. An aboveprocessing flow is an operation when an inter picture prediction codingis selected, however, a switch 111 switches to an intra pictureprediction coding. Eventually, variable length coding is performed forcontrol information, such as a motion vector, and picture information,such as picture information outputted from the predictive residualcoding unit 102, and a bit stream outputted eventually is generated by abit stream generating unit 103.

A summary of coding flow is explained above, however, details of theprocess in the motion vector detecting unit 106 and the mode selectingunit 107 are explained below.

Motion vector detecting is performed on a block-by-block basis or anarea-by-area (area is a divided block) basis. Using coded picturespreceding and following a current picture to be coded in display orderas reference pictures, a predictive picture and a prediction modeshowing a location which is predicted to be optimum in the search areain the picture is generated by deciding a motion vector.

A direct mode is one of bi-predictions which perform inter pictureprediction coding prediction referring to two pictures preceding and/orfollowing in display order. In the direct mode, a current block to becoded does not contain a motion vector directly, and two motion vectorsfor actual motion compensation are calculated referring to a motionvector of a co-located block in a coded picture close in display order,and a predictive block is generated.

FIG. 7 shows an operation when a coded block referred to in order todetermine a motion vector in the direct mode contains two motion vectorswhich refer to two pictures preceding in display order. A picture P23 isa current picture to be coded, and performs bi-prediction referring topictures P22 and P24. Assume that a block to be coded is a block MB21;and two required motion vectors are determined using a motion vectorcontained in a block MB22, which is a co-located block in the codedfollowing reference picture (the second reference picture specified bythe second reference index) P24. Since the block MB22 contains motionvectors MV21 and MV22 as a motion vector, it is impossible to calculatetwo required motion vectors MV23 and MV24 by scaling directly similarlyto Equation 1. Therefore, as Equation 2 shows, a motion vector MV_REF iscalculated as a motion vector to be scaled from an average value of twomotion vectors contained in the block MB22, and a time interval TR_REFat that time is calculated from the average value likewise. Then, motionvectors MV23 and MV24 are calculated by scaling the motion vector MV_REFand the time interval TR_REF based on Equation 3. In this case, the timeinterval TR21 indicates a time interval between the picture P24 and thepicture P21, that is, a picture referred to by the motion vector MV21,and the time interval TR22 indicates a time interval until a picturereferred to by the motion vector MV22. Moreover, the time interval TR23is a time interval until a picture referred to by the motion vectorMV23, and the time interval TR24 is a time interval until a picturereferred to by the motion vector MV24. Time intervals between thesepictures can be determined based on, for example, information indicatingdisplay time and display order added to each picture or difference ofinformation. Note that a current picture to be coded refers to a nextpicture in the example of FIG. 7, however, the case referring to apicture which is not next may be treated in the same manner.MV_REF=(MV21+MV22)/2  Equation 2 (a)TR_REF=(TR21+TR22)/2  Equation 2 (b)MV23=MV_REF/TR_REF×TR23  Equation 3 (a)MV24=−MV_REF/TR_REF×TR24  Equation 3 (b)

The above embodiment shows the coding method in which an inter pictureprediction coding can be performed using the direct mode withoutcontradiction even if a block of which motion vector is referred to inthe direct mode belongs to a B picture. In the coding method, when ablock of which motion vector is referred to in the direct mode containsplural motion vectors which refer to a picture preceding in displayorder, one motion vector is generated using the plural motion vectors,and two motion vectors to be used for actual motion compensation aredetermined by scaling.

Note that when two motion vectors MV23 and MV24 in FIG. 7 arecalculated, it is possible to use Equation 4 instead of Equation 2 as amethod for averaging motion vectors MV21 and MV22, and for averagingtime intervals TR21 and TR22 in order to calculate the motion vectorMV_REF and the time interval TR_REF to be scaled. First, as Equation 4(a) shows, the motion vector MV21′ is calculated by scaling MV21 toequate the time interval with the motion vector MV22. Then the motionvector MV_REF is determined by averaging motion vectors MV21′ and MV22.Here, the time interval TR22 is used directly as the time intervalTR_RF. Note that the case calculating a motion vector MV22′ by scalingthe motion vector MV22 instead of calculating the motion vector MV21′ byscaling a motion vector MV21 may be treated in the same manner.MV21′=MV21/TR21×TR22  Equation 4 (a)MV_RF=(MV21′+MV22)/2  Equation 4 (b)TR_REF=TR22  Equation 4 (c)

Note that when two motion vectors MV23 and MV24 in FIG. 7 arecalculated, as a motion vector MV_REF and a time interval TR_REF whichare scaled, a motion vector MV22 and a time interval TR22, which referto a picture P22 located temporally closer to a picture P24 of whichmotion vector is referred to, can be directly used as Equation 5 showsinstead of using an average value of two motion vectors as Equation 2shows. Likewise, as a motion vector MV_REF and a time interval TR_REF, amotion vector MV21 and a time interval TR21, which refer to a pictureP21 located temporally farther, can be directly used as Equation 6shows. This makes it possible to reduce capacity of a motion vectorstoring unit in a coding apparatus since each block belonging to apicture P24 of which motion vector is referred to can perform motioncompensation by storing only one of two motion vectors.MV_REF=MV22  Equation 5 (a)TR_REF=TR22  Equation 5 (b)MV_REF=MV21  Equation 6 (a)TR_REF=TR21  Equation 6 (b)

Note that when two motion vectors MV23 and MV24 in FIG. 7 arecalculated, as a motion vector MV_REF and a time interval TR_REF whichare scaled, a motion vector which refers to a picture to be codedprecedently can be directly used instead of using an average value oftwo motion vectors as Equation 2 shows. FIG. 8A shows a referencerelation in display order of moving pictures as FIG. 7 shows, and FIG.8B shows an example of an order in which pictures are reordered bycoding order in the frame memory 101 shown in FIG. 6. Here, a pictureP23 indicates a picture to be coded in the direct mode, and a pictureP24 indicates a picture of which motion vector is referred to for thecoding. When pictures are reordered as shown in FIG. 8B, since a motionvector which refers to a picture to be coded precedently is directlyused, a motion vector MV22 and a time interval TR22 are directly used asa motion vector MV_REF and a time interval TR_REF as shown in Equation5. Likewise, it is possible to directly use a motion vector which refersto a picture to be coded later. In this case, a motion vector MV21 and atime interval TR21 are directly applied as a motion vector MV_REF and atime interval TR_REF as Equation 6 shows. This make it possible toreduce capacity of a motion vector storing unit in a coding apparatussince each block belonging to a picture P24 of which motion vector isreferred to can perform motion compensation by storing only one of twomotion vectors.

Note that in this embodiment, the case that a motion vector used in thedirect mode is calculated by scaling a referenced motion vector using atime interval between pictures is explained, however, the motion vectormay be calculated by multiplying by a constant number. Here, a constantused for the multiplication may be variable when coding or decoding isperformed on plural blocks basis or on plural pictures basis.

Note that in Equation 2 (a) or 4 (b), when a motion vector MV_REF iscalculated, after calculating the right side of Equation 2 (a) or 4 (b),the motion vector may be rounded to a predetermined motion vectorprecision (for example, round to a value of 0.5 pixel unit for a motionvector with half pixel precision). Precision of a motion vector is notlimited to half pixel precision. In addition, precision of a motionvector can be determined on block basis, picture basis, and sequencebasis, for example. Moreover, in Equations, 3 (a), 3 (b) and 4 (a), whenmotion vectors MV23, MV24 and MV21′ are calculated, motion vectors maybe rounded to a predetermined precision of a motion vector aftercalculating the right side of Equations 3 (a), 3 (b) and 4 (a).

Second Embodiment

An overview of a coding process based on FIG. 6 is completely equal tothe first embodiment. Here, a detailed operation of bi-prediction in thedirect mode is explained using FIG. 9.

FIG. 9 shows an operation when a block referred to in order to determinea motion vector in the direct mode contains two motion vectors whichrefer to two following pictures in display order. A picture P43 is acurrent picture to be coded, and performs bi-prediction referring topictures P42 and P44. Assume that a block to be coded is a block MB41,then two required motion vectors are determined using a motion vector ofa co-located block MB42 in the coded backward reference picture (thesecond reference picture specified by the second reference index) P44.Since the block MB42 contains two motion vectors MV45 and MV46 as motionvectors, two required motion vectors MV43 and MV44 cannot be calculatedby applying directly scaling similarly to Equation 1. Therefore, asEquation 7 shows, a motion vector MV_REF is determined as a motionvector to be scaled from an average value of two motion vectors of theblock MB42, and a time interval TR_REF at that time is determined froman average value likewise. Then motion vectors MV43 and MV44 arecalculated by scaling a motion vector MV_REF and a time interval TR_REFbased on Equation 8. In this case, a time interval TR45 indicates a timeinterval between a picture P44 and P45, that is, until a picture whichis referred to by a motion vector MV45; and a time interval TR46indicates a time interval until a picture which is referred to by amotion vector MV46. A time interval TR43 indicates a time interval untila picture which is referred to by a motion vector MV43; and a timeinterval TR44 indicates a time interval until a picture which isreferred to by a motion vector MV44. Time intervals between thesepictures can be determined based on, for example, information indicatingdisplay time and display order that is added to each picture ordifference of information as explained in the first embodiment. Notethat a current picture to be coded refers to a next picture in theexample of FIG. 9, however, the case referring to a picture which is notnext may be treated in the same manner.MV_REF=(MV45+MV46)/2  Equation 7 (a)TR_REF=(TR45+TR46)/2  Equation 7 (b)MV43=−MV_REF/TR_REF×TR43  Equation 8 (a)MV44=MV_REF/TR_REF×TR44  Equation 8 (b)

The above embodiment shows the coding method in which an inter pictureprediction coding can be performed using the direct mode withoutcontradiction even if a block of which motion vector is referred to inthe direct mode belongs to a B picture. In the coding method, when ablock of which motion vector is referred to in the direct mode containsplural motion vectors which refer to a following picture in displayorder, a motion vector is generated using the plural motion vectors, andtwo motion vectors to be used for actual motion compensation aredetermined by scaling.

Note that when two motion vectors MV43 and MV44 in FIG. 9 arecalculated, it is possible to use Equation 9 instead of Equation 7 as amethod for averaging motion vectors MV45 and MV46 and for averaging timeintervals TR45 and TR46 in order to calculate the motion vector MV_REFand the time interval TR_REF to be scaled. First, as Equation 9 (a)shows, the motion vector MV46′ is calculated by scaling MV46 to equatethe time interval with the motion vector MV45. Then the motion vectorMV_REF is determined by averaging motion vectors MV46′ and MV45. Here,the time interval TR41 is used directly as the time interval TR_REF.Note that the case calculating a motion vector MV45′ by scaling themotion vector MV45 instead of calculating the motion vector MV46′ byscaling a motion vector MV46 may be treated in the same manner.MV46′=MV46/TR46×TR45  Equation 9 (a)MV_REF=(MV46′+MV45)/2  Equation 9 (b)TR_REF=TR45  Equation 9 (c)

Note that when two motion vectors MV43 and MV44 in FIG. 9 arecalculated, as a motion vector MV_REF and a time interval TR_REF whichare scaled, a motion vector MV45 and a time interval TR45, which referto a picture P45 located temporally closer to a picture P44 of whichmotion vector is referred to, can be directly used as Equation 10 showsinstead of using an average value of two motion vectors as Equation 7shows. Likewise, as a motion vector MV_REF and a time interval TR_REF, amotion vector MV46 and a time interval TR46, which refer to a pictureP46 located temporally farther can be directly used as Equation 11shows. This method makes it possible to reduce capacity of a motionvector storing unit in a coding apparatus since each block belonging toa picture P44 of which motion vector is referred to can implement motioncompensation by storing only one of two motion vectors.MV_REF=MV45  Equation 10 (a)TR_REF=TR45  Equation 10 (b)MV_REF=MV46  Equation 11 (a)TR_REF=TR46  Equation 11 (b)

Note that when two motion vectors MV43 and MV44 in FIG. 9 arecalculated, as a motion vector MV_REF and a time interval TR_REF whichare scaled, a motion vector which refers to a picture to be codedprecedently can be directly used instead of using an average value oftwo motion vectors as Equation 7 shows. FIG. 10A shows a referentialrelation of pictures in display order of moving pictures as FIG. 9shows, and FIG. 10B shows an example of an order in which pictures arereordered in coding order in the frame memory 101 shown in FIG. 6. Here,a picture P43 indicates a picture to be coded in the direct mode, and apicture P44 indicates a picture of which motion vector is referred tofor the coding. When pictures are reordered as shown in FIG. 10B, sincea motion vector which refers to a picture to be coded precedently isdirectly used, a motion vector MV46 and a time interval TR46 aredirectly used as a motion vector MV_REF and a time interval TR_REF asshown in Equation 11. Likewise, it is possible to directly use a motionvector which refers to a picture to be coded later. In this case, amotion vector MV45 and a time interval TR45 are directly applied as amotion vector MV_REF and a time interval TR_REF. This method makes itpossible to reduce capacity of a motion vector storing unit in a codingapparatus since each block belonging to a picture P44 of which motionvector is referred to can perform motion compensation by storing onlyone of two motion vectors.

Note that when a picture which is referred to in order to determine amotion vector in the direct mode contains two motion vectors which referto two following pictures in display order, it is possible to performmotion compensation assuming that two required motion vectors MV43 andMV44 are “0”. This method makes it possible to reduce capacity of amotion vector storing unit in a decoding apparatus, and further makes itpossible to omit a process of calculating a motion vector, since eachblock belonging to a picture P44 of which motion vector is referred todoes not have to store a motion vector.

Note that when a picture which is referred to in order to determine amotion vector in the direct mode contains two motion vectors which referto two following pictures in display order, it is possible to inhibitreferring to a motion vector and to apply only a prediction coding otherthan the direct mode. When following two pictures in display order arereferred to as a picture P44 shown in FIG. 9, it is conceivable thatcorrelation with a preceding picture in display order is low, because itis possible to generate a more precise predictive picture by inhibitingthe direct mode and selecting other predicting method.

Note that in this embodiment, the case that a motion vector used in thedirect mode is calculated by scaling a referenced motion vector using atime interval between pictures is explained, however, the motion vectormay be calculated by multiplying by a constant number. Here, a constantused for the multiplication may be variable when coding or decoding isperformed on plural blocks basis or on plural pictures basis.

Note that in Equation 7 (a) or 9 (b), when a motion vector MV_REF iscalculated, after calculating the right side of Equation 7 (a) or 9 (b),the motion vector may be rounded to a predetermined motion vectorprecision. Precision of a motion vector includes half pixel precision,one-third pixel precision and quarter pixel precision or the like. Inaddition, the precision of a motion vector can be determined, forexample, on block basis, picture basis, and sequence basis. Moreover, inEquations, 8 (a), 8 (b) and 9 (a), when motion vectors MV43, MV44 andMV46′ are calculated, motion vectors may be rounded to a predeterminedprecision of a motion vector after calculating the right side ofEquations 8 (a), 8 (b) and 9 (a).

Third Embodiment

A moving picture decoding method according to the third embodiment ofthe present invention is explained using the block diagram shown in FIG.11. However, it is assumed that the bit stream generated in the picturecoding method of the first embodiment is inputted.

First, various information such as a prediction mode, motion vectorinformation and predictive residual coding data is extracted frominputted bit stream by a bit stream analyzer 601.

The prediction mode and the motion vector information are outputted to aprediction mode/motion vector decoding unit 608 and a predictiveresidual coding data is outputted to a predictive residual decoding unit602. The prediction mode/motion compensation decoding unit 608 decodesthe prediction mode and a motion vector used in the prediction mode.When decoding the motion vector, a decoded motion vector stored in themotion vector storing unit 605 is used. Decoded prediction mode andmotion vector are outputted to a motion compensation decoding unit 604.In addition, decoded motion vector is stored in the motion vectorstoring unit 605 in order to be used for decoding motion vectors offollowing blocks. In the motion compensation decoding unit 604, apredictive picture is generated based on the inputted prediction modeand motion vector information using a decoded picture stored in a framememory 603 as a reference picture. A decoded picture is generated byinputting the above generated predictive picture into an add operatingunit 606 and adding the inputted picture to the predictive residualpicture generated in a predictive residual decoding unit 602. The aboveembodiment shows an operation for an inter-picture-prediction-bitstream, however, a switch 607 switches to a decoding process for anintra-picture-prediction-bit stream.

A summary of a decoding flow is shown above, however, a detailed processin the motion compensation decoding unit 604 is explained below.

Motion vector information is added on a block basis or an area (adivided block) basis. By using decoded pictures preceding and followinga current picture to be coded in display order as reference pictures, apredictive picture to perform motion compensation from the pictures isgenerated.

A direct mode is one of bi-predictions which perform inter pictureprediction coding referring to each of pictures preceding and followingin display order. In the direct mode, since a current block to be codedinputs a bit stream which does not contain a motion vector directly, twomotion vectors for actual motion compensation are calculated referringto a motion vector of a co-located block in a decoded picture close indisplay order, and a predictive picture is generated.

FIG. 7 shows an operation when a decoded picture referred to in order todetermine a motion vector in the direct mode contains two motion vectorswhich refer to preceding two pictures in display order. A picture P23 isa current picture to be decoded, and performs bi-prediction referring topictures P22 and P24. When it is assumed that a block to be decoded is ablock MB21, two required motion vectors are determined using a motionvector of a co-located block MB22 in the decoded backward referencepicture (the second reference picture specified by the second referenceindex) P24. Since the block MB22 contains two motion vectors MV21 andMV22 as the motion vectors, two required motion vectors MV23 and MV24cannot be calculated by applying the direct scaling similarly toEquation 1. Therefore, as Equation 2, a motion vector MV_REF isdetermined as a motion vector to be scaled from an average value of twomotion vectors of the block MB22, and a time interval TR_REF at thattime is determined from an average value likewise. Then motion vectorsMV23 and MV24 are calculated by scaling a motion vector MV_REF and atime interval TR_REF based on Equation 3. In this case, a time intervalTR21 indicates a time interval between a picture P24 and P21, that is,until a picture which is referred to by a motion vector MV21, and a timeinterval TR22 indicates a time interval until a picture which isreferred to by a motion vector MV22. A time interval TR23 indicates atime interval until a picture which is referred to by a motion vectorMV23; and a time interval TR24 indicates a time interval until a picturewhich is referred to by a motion vector MV24. Time intervals betweenthese pictures can be determined based on, for example, informationindicating display time and display order added to each picture ordifference of information. Note that a current picture to be codedrefers to a next picture in the example of FIG. 7, however, the casereferring to a picture which is not next may be treated in the samemanner.

The above embodiment shows the decoding method in which an inter pictureprediction decoding can be performed using the direct mode withoutcontradiction even if a block of which motion vector is referred tobelongs to a B picture. In the decoding method, when a block of whichmotion vector is referred to in the direct mode contains plural motionvectors which refer to a preceding picture, a motion vector is generatedusing the plural motion vectors, and two motion vectors to be used foractual motion compensation are determined by scaling.

Note that when two motion vectors MV23 and MV24 in FIG. 7 arecalculated, it is possible to use Equation 4 instead of Equation 2 as amethod for averaging motion vectors MV21 and MV22 and for averaging timeintervals TR21 and TR22 in order to calculate the motion vector MV_REFand the time interval TR_REF to be scaled. First, as Equation 4 (a)shows, the motion vector MV21′ is calculated by scaling MV21 to equatethe time interval with the motion vector MV22. Then the motion vectorMV_REF is determined by averaging motion vectors MV21′ and MV22. Here,the time interval TR22 is used directly as the time interval TR_REF.Note that the case calculating a motion vector MV22′ by scaling themotion vector MV22 instead of calculating the motion vector MV21′ byscaling a motion vector MV21 may be treated in the same manner.

Note that when two motion vectors MV23 and MV24 in FIG. 7 arecalculated, as a motion vector MV_REF and a time interval TR_REF whichare scaled, a motion vector MV22 and a time interval TR22, which referto a picture P22 located temporally closer to a picture P24 of whichmotion vector is referred to, can be directly used as Equation 5 showsinstead of using an average value of two motion vectors as Equation 2shows. Likewise, as a motion vector MV_REF and a time interval TR_REF, amotion vector MV21 and a time interval TR21, which refer to a pictureP21 located temporally farther can be directly used as Equation 6 shows.This method makes it possible to reduce capacity of a motion vectorstoring unit in a coding apparatus since each block belonging to apicture P24 of which motion vector is referred to can actualize motioncompensation by storing only one of two motion vectors.

Note that when two motion vectors MV23 and MV24 in FIG. 7 arecalculated, as a motion vector MV_REF and a time interval TR_REF whichare scaled, a motion vector which refers to a picture to be decodedprecedently can be directly used, instead of using an average value oftwo motion vectors as Equation 2 shows. FIG. 8A shows a referentialrelation in display order of moving pictures as FIG. 7 shows, and FIG.8B shows an order in which a bit stream is inputted, that is, a decodingorder. Here, a picture P23 indicates a picture decoded in the directmode, and a picture P24 indicates a picture of which motion vector isreferred to for the decoding. When considering an order as shown in FIG.8B, since a motion vector which refers to a picture to be decodedprecedently is directly used, a motion vector MV22 and a time intervalTR22 are directly applied as a motion vector MV_REF and a time intervalTR_REF as Equation 5 shows. Likewise, it is possible to directly use amotion vector which refers to a picture to be decoded later. In thiscase, a motion vector MV21 and a time interval TR21 are directly appliedas a motion vector MV_REF and a time interval TR_REF as Equation 6shows. This makes it possible to reduce capacity of a motion vectorstoring unit in a decoding apparatus since each block belonging to apicture P24 of which motion vector is referred to can perform motioncompensation by storing only one of two motion vectors.

Note that in this embodiment, the case that a motion vector used in thedirect mode is calculated by scaling a referenced motion vector using atime interval between pictures is explained, however, the motion vectormay be calculated by multiplying by a constant number. Here, a constantused for the multiplication may be variable when coding or decoding isperformed on plural blocks basis or on plural pictures basis.

Fourth Embodiment

An overview of a coding process based on FIG. 11 is completely equal tothe third embodiment. Here, a detailed operation of bi-prediction in thedirect mode is explained using FIG. 9. However, it is assumed that thebit stream generated in the picture coding method of the firstembodiment is inputted.

FIG. 9 shows an operation when a picture referred to in order todetermine a motion vector in the direct mode contains two motion vectorswhich refer to following two pictures in display order. A picture P43 isa current picture to be decoded, and performs bi-prediction referring topictures P42 and P44. When it is assumed that a block to be decoded is ablock MB41, two required motion vectors are determined using a motionvector of a co-located block MB42 in the decoded backward referencepicture (the second reference picture specified by the second referenceindex) P44. Since the block MB42 contains two motion vectors MV45 andMV46 as motion vectors, two motion vectors MV43 and MV44 cannot becalculated by directly scaling similarly to Equation 1. Therefore, asEquation 7, a motion vector MV_REF is determined as a motion vector tobe scaled from an average value of two motion vectors of the block MB42,and a time interval TR_REF at that time is determined from an averagevalue likewise. Then motion vectors MV43 and MV44 are calculated byscaling a motion vector MV_REF and a time interval TR_REF based onEquation 8. In this case, a time interval TR45 indicates a time intervalbetween a picture P44 and P45, that is, until a picture which isreferred to by a motion vector MV45; and a time interval TR46 indicatesa time interval between until a picture which is referred to by a motionvector MV46. A time interval TR43 indicates a time interval until apicture which is referred to by a motion vector MV43; and a timeinterval TR44 indicates a time interval until a picture which isreferred to by a motion vector MV44. Note that a current picture to bedecoded refers to a next picture in the example of FIG. 9, however, thecase referring to a picture which is not adjacent may be treated in thesame manner.

The above embodiment shows the decoding method in which an inter pictureprediction decoding can be performed using the direct mode withoutcontradiction even if a block of which motion vector is referred to inthe direct mode belongs to a B picture. In the decoding method, when ablock of which motion vector is referred to in the direct mode containsplural motion vectors which refer to a following picture in displayorder, a motion vector is generated using the plural motion vectors, andtwo motion vectors to be used for actual motion compensation aredetermined by scaling.

Note that when two motion vectors MV43 and MV44 in FIG. 9 arecalculated, it is possible to use Equation 7 instead of Equation 9 as amethod for averaging motion vectors MV45 and MV46 and for averaging timeintervals TR45 and TR46 in order to calculate the motion vector MV_REFand the time interval TR_REF to be scaled. First, as Equation 9 (a)shows, the motion vector MV46′ is calculated by scaling MV46 to equatethe time interval with the motion vector MV45. Then the motion vectorMV_REF is determined by averaging motion vectors MV46′ and MV45. Here,the time interval TR45 is used directly as the time interval TR_RF. Notethat the case calculating a motion vector MV45′ by scaling the motionvector MV45 instead of calculating the motion vector MV46′ by scaling amotion vector MV46 may be treated in the same manner.

Note that when two motion vectors MV43 and MV44 in FIG. 9 arecalculated, as a motion vector MV_REF and a time interval TR_REF whichare scaled, a motion vector MV45 and a time interval TR45, which referto a picture P45 located temporally closer to a picture P44 of whichmotion vector is referred to, can be directly used, as Equation 10shows, instead of using an average value of two motion vectors asEquation 7 shows. Likewise, as a motion vector MV_REF and a timeinterval TR_REF, a motion vector MV46 and a time interval TR46, whichrefer to a picture P46 located temporally farther can be directly usedas Equation 11 shows. This method makes it possible to reduce capacityof a motion vector storing unit in a decoding apparatus since each blockbelonging to a picture P44 of which motion vector is referred to canimplement motion compensation by storing only one of two motion vectors.

Note that when two motion vectors MV43 and MV44 in FIG. 9 arecalculated, as a motion vector MV_REF and a time interval TR_REF whichare scaled, a motion vector which refers to a picture to be decodedprecedently can be directly used, instead of using an average value oftwo motion vectors as Equation 7 shows. FIG. 10A shows a referentialrelation in display order of moving pictures as FIG. 9 shows and FIG.10B shows an order in which a bit stream is inputted, that is, adecoding order. Here, a picture P43 indicates a picture which is decodedin the direct mode, and a picture P44 indicates a picture of whichmotion vector is referred to for the decoding. When considering an orderas shown in FIG. 10B, since a motion vector which refers to a picture tobe decoded precedently is directly used, a motion vector MV46 and a timeinterval TR46 are directly applied as a motion vector MV_REF and a timeinterval TR_REF as Equation 10 shows. This method makes it possible toreduce capacity of a motion vector storing unit in a decoding apparatussince each block belonging to a picture P44 of which motion vector isreferred to can perform motion compensation by storing only one of twomotion vectors.

Note that when a block which is referred to in order to determine amotion vector in the direct mode contains two motion vectors which referto two following pictures in display order, it is possible to performmotion compensation assuming that two required motion vectors MV43 andMV44 are “0”. This method makes it possible to reduce capacity of amotion vector storing unit in a decoding apparatus and further makes itpossible to omit a process of calculating a motion vector since eachblock belonging to a picture P44 of which motion vector is referred todoes not have to store a motion vector.

Note that in this embodiment, the case that a motion vector used in thedirect mode is calculated by scaling a referenced motion vector using atime interval between pictures is explained, however, the motion vectormay be calculated by multiplying by a constant number. Here, a constantused for the multiplication may be variable when coding or decoding isperformed on plural blocks basis or on plural pictures basis.

Fifth Embodiment

Coding/decoding method can be actualized not only by the coding/decodingmethod shown in the above first embodiment through fourth embodiment,but also by a motion vector calculation method shown below.

FIG. 12 shows an operation when a coded or decoded block referred to inorder to calculate a motion vector in the direct mode contains twomotion vectors which refer to preceding two pictures in display order. Apicture P23 is a current picture to be coded or decoded. When it isassumed that a block to be coded or decoded is a block MB1, two requiredmotion vectors are determined using a motion vector of a co-locatedblock MB2 in the coded or decoded backward reference picture (the secondreference picture specified by the second reference index) P24. Notethat in FIG. 12, the block MB1 is a current block of process, the blocksMB1 and MB2 are co-located blocks in other pictures, and the motionvectors MV21 is first forward motion vector that the reference pictureis specified by first reference index and MV22 is forward motion vectorthat the reference picture is specified by second reference index, andthese motion vectors are used for coding or decoding the block MB2 andrefer to pictures P21 and P22 respectively. The pictures P21, P22 andP24 are coded or decoded pictures. A time interval TR21 is a timeinterval between the picture P21 and the picture P24; a time intervalTR22 is a time interval between the picture P22 and the picture P24; atime interval TR21′ is a time interval between P21 and the picture P23;and a time interval TR 24′ is a time interval between the picture P23and the picture P24.

In the motion vector calculation method, as shown in FIG. 12, only theforward motion vector (the first motion vector) MV21 coded or decodedprecedently is used out of motion vectors of the block MB2 in thereference picture P24, and a motion vectors MV21′ and MV24′ of the blockMB1 are calculated by following equations.MV21′=MV21×TR21′/TR21MV24′=−MV21×TR24′/TR21

Then bi-prediction is performed from the pictures P21 and P24 using themotion vectors MV21′ and MV24′. Note that a motion vector of the blockMB1 may be calculated using only a motion vector (the second motionvector) MV22 coded or decoded later out of motion vectors of the blockMB2 in the reference picture P24, instead of calculating motion vectorsMV21′ and MV24′ of the block MB1 using only the motion vector MV21.Moreover, as shown in the first embodiment through the fourthembodiment, a motion vector of the block MB1 may be determined usingboth the motion vectors MV21 and MV22. When selecting one of the motionvectors MV21 and MV22, a motion vector of a block coded or decodedprecedently may be selected, and it may be set arbitrarily in a codingapparatus and a decoding apparatus. Motion compensation is possibleeither when the picture P21 is in the short term reference picturebuffer or in the long term reference picture buffer. Explanation will begiven for the short term reference picture buffer and the long termreference picture buffer later.

FIG. 13 shows an operation when a coded or decoded block referred to inorder to calculate a motion vector in the direct mode contains twomotion vectors which refer to following two pictures in display order. Apicture P22 is a current picture to be coded or decoded. When it isassumed that a block to be coded or decoded is a block MB1, two requiredmotion vectors are determined using a motion vector of a co-locatedblock MB2 in the coded or decoded backward reference picture (the secondreference picture) P23. Note that in FIG. 13 the block MB1 is a currentblock of processing, the blocks MB1 and MB2 are co-located blocks inpictures, and the motion vectors MV24 and MV25 are backward motionvectors used for coding or decoding the block MB2 and refer to picturesP21 and P22, respectively. The pictures P21, P23, P24 and P25 are codedor decoded pictures. A time interval TR24 is a time interval between thepicture P23 and the picture P24, a time interval TR25 is a time intervalbetween the picture P23 and the picture P25, a time interval TR24′ is atime interval between P22 and the picture P24, and a time interval TR21′ is a time interval between the picture P21 and the picture P22.

In a motion vector calculation method, as shown in FIG. 13, only thebackward motion vector MV24, which refers to the picture P24, of theblock MB2 in the reference picture P23 is used, and a motion vectorsMV21′ and MV24′ are calculated by following equations.MV21′=−MV24×TR21′/TR24MV24′=MV24×TR24′/TR24

Then bi-prediction is performed from the pictures P21 and P24 using themotion vectors MV21′ and MV24′.

Note that, as shown in FIG. 14, when only a backward motion vector MV25, which points at the picture P25, of the block MB2 in the referencepicture P23 is used, motion vectors MV21′ and MV24′ are calculated byfollowing equations. Here, a time interval TR24 is a time intervalbetween the picture P23 and the picture P24; a time interval TR25 is atime interval between the picture P23 and the picture P25; a timeinterval TR25′ is a time interval between the picture P22 and thepicture P25; and a time interval TR21′ is a time interval between thepicture P21 and the picture P22.MV21′=−MV25×TR21′/TR25MV25′=MV25×TR25′/TR25

Then bi-prediction is performed from the pictures P21 and P24 using themotion vectors MV21′ and MV24′.

FIG. 15 shows an operation when a coded or decoded block referred to inorder to calculate a motion vector in the direct mode contains twomotion vectors which refer to a preceding picture in display order. Apicture P23 is a current picture to be coded or decoded. When it isassumed that a block to be coded or decoded is a block MB1, two requiredmotion vectors are determined using a motion vector of a co-locatedblock MB2 in the coded or decoded backward reference picture (the secondreference picture specified by the second reference index) P24. Notethat in FIG. 15, the block MB1 is a current block of processing, theblocks MB1 and MB2 are co-located blocks in other pictures. The motionvectors MV21A and MV21B are forward motion vectors used for coding ordecoding the block MB2, and both refer to the picture P21. The picturesP21, P22 and P24 are coded or decoded pictures. Time intervals TR21A andTR21B are a time interval between the picture P21 and the picture P24; atime interval TR21′ is a time interval between the picture P21 and thepicture P23; and a time interval TR24′ is a time interval between P23and the picture P24.

In a motion vector calculation method, as shown in FIG. 15, only theforward motion vector MV21A, which points at the picture P21, of theblock MB2 in the reference picture P24 is used, and a motion vectorsMV21A′ and MV24′ are calculated by following equations.MV21A′=MV21A×TR21′/TR21AMV24′=−MV21A×TR24′/TR21A

Then bi-prediction is performed from the pictures P21 and P24 using themotion vectors MV21A′ and MV24′.

Note that a motion vector of the block MB1 may be calculated using onlya forward motion vector MV21B, which points at the picture P21, of theblock MB2 in the reference picture P24. Moreover, as shown in the firstembodiment through the fourth embodiment, a motion vector of the blockMB1 may be determined using both forward motion vectors MV21A and MV21B.When selecting one of the forward motion vectors MV21A and MV21B, amotion vector of a block coded or decoded precedently (described earlierin a bit stream) may be selected, and it may be set arbitrarily by acoding apparatus and a decoding apparatus. Here, the motion vector codedor decoded precedently means the first motion vector. Motioncompensation is possible either when the picture P21 in the short termreference picture buffer or in the long term reference picture buffer.Explanation will be given for the short term reference picture bufferand the long term reference picture buffer later.

Note that in this embodiment, the case that a motion vector used in thedirect mode is calculated by scaling a referenced motion vector using atime interval between pictures is explained, however, the motion vectormay be calculated by multiplying by a constant number. Here, a constantused for the multiplication may be variable when coding or decoding isperformed on plural blocks basis or on plural pictures basis.

Note that, in the above-mentioned equations to calculate motion vectorsMV21′, MV24′, MV25′ and MV21A′, motion vectors may be rounded to apredetermined precision of a motion vector after calculating the rightside of the equations. Precision of motion vector includes half pixelprecision, one-third pixel precision and quarter pixel precision or thelike. In addition, precision of a motion vector can be determined, forexample, on block basis, picture basis, and sequence basis.

Sixth Embodiment

In this sixth embodiment, a method for calculating a current motionvector by scaling only one of two forward motion vectors, which refer totwo pictures preceding in display order, is explained using FIGS. 14, 15and 16. In this case, a reference picture used to in order to determinea current motion vector in the direct mode contains the two forwardmotion vectors. Note that the block MB1 is a current block to beprocessed; the blocks MB1 and MB2 are co-located blocks in otherpictures; and the motion vectors MV21 and MV22 are forward motionvectors used for coding or decoding the block MB2, and refer to picturesP21 and P22, respectively. The pictures P21, P22 and P24 are coded ordecoded pictures. A time interval TR21 is a time interval between thepicture P21 and the picture P24; a time interval TR22 is a time intervalbetween the picture P22 and the picture P24; a time interval TR21′ is atime interval between P21 and the picture P23; and a time interval TR22′is a time interval between P22 and the picture P23.

As the first method, when a block MB2 in a reference picture P24contains a forward motion vector MV21 referring to a picture P22 and aforward motion vector MV22 referring to a picture P23 as shown in FIG.16, a motion vector MV22′ of the block MB1 is calculated using only amotion vector MV22 which refers to a picture P22 close to a currentpicture P23 in display order by a following equation.MV22′=MV22×TR22′/TR22

Then motion compensation is performed from the picture P22 using themotion vector MV22′.

As the second method, when a block MB2 of a reference picture P24contains a forward motion vector MV21 referring to a picture P21 and aforward motion vector MV22 referring to a picture P22 as shown in FIG.17, a motion vector MV21′ of the block MB1 is calculated using only amotion vector MV21 which refers to a picture P21 being far from acurrent picture P23 in display order by a following equation.MV21′=MV21×TR21′/TR21

Then motion compensation is performed from the picture P21 using themotion vector MV21′.

The first and the second methods make it possible to reduce capacity ofa motion vector storing unit since the block MB2 belonging to a pictureP24 of which motion vector is referred to can actualize motioncompensation by storing only one of two motion vectors.

Note that motion compensation can be performed from a picture P22 closein display order, using the forward motion vector MV21 same as the firstembodiment. A motion vector MVN (not shown in this figure) used for themotion compensation is calculated by a following equation.MVN=MV21×TR22′/TR21

As the third method, as shown in FIG. 18, a motion compensation block isobtained from the pictures P21 and P22 respectively using the motionvectors MV21′ and MV22′ calculated above, and an average picture is usedas an interpolation picture in motion compensation.

The third method increases calculated amount, however, improvesprecision of motion compensation.

Moreover, it is possible to obtain a motion compensation block from thepicture P22 using the above-mentioned motion vectors MVN and MV22′, andto use an average picture as an interpolation picture in motioncompensation.

Note that in this embodiment, the case that a motion vector used in thedirect mode is calculated by scaling a referenced motion vector using atime interval between pictures is explained, however, the motion vectormay be calculated by multiplying a reference motion vector by a constantnumber. Here, a constant used for the multiplication may be variablewhen coding or decoding is performed on plural blocks basis or on pluralpictures basis.

Note that, in the above-mentioned equations to calculate the motionvectors MV21′, MV22′ and MVN, the motion vectors may be rounded to apredetermined precision of a motion vector after calculating the rightside of the equations. Precision of motion vector includes half pixelprecision, one-third pixel precision and quarter pixel precision or thelike. In addition, precision of a motion vector can be determined, forexample, on block basis, picture basis, and sequence basis.

Seventh Embodiment

In the above sixth embodiment, the case when a reference picture used todetermine a motion vector of a current block to be coded or decodedcontains two forward motion vectors in the direct mode is described. Thetwo forward motion vectors refer to two preceding pictures in displayorder. However, when the reference picture contains two backward motionvectors which refer to two following pictures in display order, it ispossible to calculate a current motion vector by scaling only one of twobackward motion vectors (the second motion vectors of which referencepicture is specified by the second reference indices), likewise.Explanation will be given using FIGS. 17˜20 below. Note that the blockMB1 is a current block of process, the blocks MB1 and MB2 are co-locatedblocks in other pictures, and the motion vectors MV24 and MV25 arebackward motion vectors (the second motion vectors of which referencepicture is specified by the second reference indices) used for coding ordecoding the block MB2. The pictures P21, P23, P24 and P25 are coded ordecoded pictures. A time interval TR24 is a time interval between thepicture P23 and the picture P24; a time interval TR25 is a time intervalbetween the picture P23 and the picture P25; a time interval TR24′ is atime interval between P22 and the picture P25; and a time interval TR25′is a time interval between P22 and the picture P25.

As the first method, when a block MB2 of a reference picture P23contains two backward motion vectors MV24 referring to a picture P24 andMV25 referring to a picture P25 as shown in FIG. 19, a motion vectorMV24′ of the block MB1 is calculated using only a backward motion vectorMV24 which refers to a picture P24 being temporally close to a currentpicture P22 by a following equation.MV24′=MV24×TR24′/TR24

Then motion compensation is performed from the picture P24 using themotion vector MV24′.

Note that motion compensation can be performed from a picture P23 closein display order, using a backward motion vector MV24 same as the firstembodiment. A motion vector MVN1 (not shown in this figure) used for themotion compensation is calculated by a following equation.MVN1=MV24×TRN1/TR24

As the second method, when a block MB2 of a reference picture P23contains two backward motion vectors MV24 referring to a picture P24 andMV25 referring to a picture P25 as shown in FIG. 20, a motion vectorMV25′ of the block MB1 is calculated using only a backward motion vectorMV25 which refers to a picture P25 far from a current picture P23 indisplay order by a following equation.MV25′=MV25×TR25′/TR25

Then motion compensation is performed from the picture P25 using themotion vector MV25′.

The first and the second methods make it possible to reduce capacity ofa motion vector storing unit since the block MB2 belonging to a pictureP23 of which motion vector is referred to can implement motioncompensation by storing only one of two motion vectors.

Note that a motion compensation can be performed from a picture P23close in display order, using a backward motion vector MV25 as same asthe first embodiment. A motion vector MVN2 (not shown in this figure)used for the motion compensation is calculated using a followingequation.MVN2=MV25×TRN1/TR25

As the third method, as shown in FIG. 21, a motion compensation block isobtained from the pictures P24 and P25 respectively using the motionvectors MV24′ and MV25′ calculated above, and an average picture is usedas an interpolation picture in a motion compensation.

The third method increases the amount of calculation, however, improvesprecision of motion compensation.

Note that, it is possible to obtain a motion compensation block from thepicture P24 using the above-mentioned motion vectors MVN1 and MVN2, anduse an average picture as an interpolation picture in motioncompensation.

Moreover, as shown in FIG. 22, when a reference picture referred to inorder to determine a motion vector of a current motion vector in thedirect mode contains a backward motion vector which refers to a picturefollowing in display order, for example, a motion vector MV24′ iscalculated using a following equation.MV24′=MV24×TR24′/TR24

Then motion compensation is performed from the picture P24 using themotion vector MV24′.

Note that motion compensation can be performed from a picture P23 closein display order, using a backward motion vector MV25 as same as thefirst embodiment. A motion vector MVN3 (not shown in this figure) usedfor the motion compensation is calculated by a following equation.MVN3=MV24×TRN1/TR24

Note that in this embodiment, the case when a current motion vector iscalculated by scaling the backward motion vector is explained, whencontaining two backward motion vectors, which refer to two picturesfollowing in display order, and when containing a backward motionvector, which refers to a picture following in display order. However, acurrent motion vector may be calculated referring to a motion vector ofa neighboring block in a same picture without using a backward motionvector, and when intra picture coding is performed, a current motionvector may be calculated referring to a motion vector of a neighboringblock in a same picture.

To begin with, the first calculation method will be described.

FIG. 23 shows a positional relation between a motion vector to bereferred to and a current block. A block MB 1 is a current block, andrefers to a motion vector of a block including three pixels located onA, B and C. Note that when a pixel C cannot be referred to since it islocated outside of a frame or it has not been coded/decoded, a motionvector of a block including a pixel D is used instead of a blockincluding the pixel C. By calculating a median value of motion vectorsof three current blocks including pixels A, B and C to be referred to, amotion vector used actually in the direct mode is determined. Bycalculating a median value of motion vectors of three blocks, additionalinformation showing which motion vector is selected is not necessary tobe described in a bit stream. Hence, it is possible to obtain a motionvector expressing motion close to actual motion of the block MB1. Inthis case, motion compensation may be performed by only forwardreference (reference to the first reference picture) using thedetermined motion vector and by bi-directional reference (reference tothe first reference picture and the second reference picture) using amotion vector parallel with the determined motion vector.

Next, the second calculation method will be described.

Under the second calculation method, a median value is not selected asthe first calculation method, and a motion vector used in actual directmode is determined by selecting a motion vector of which codingefficiency is the highest of motion vectors of three blocks includingpixels A, B and C. In this case, motion compensation may be performed byonly forward reference (reference to the first reference picture), usingthe determined motion vector and by bi-directional reference (referenceto the first reference picture and the second reference picture) using amotion vector parallel with the determined motion vector. Informationindicating a motion vector with the highest coding efficiency is, forexample as shown in FIG. 24A, added to a header area of a block in a bitstream generated by a bit stream generating unit 103 with informationindicating a direct mode outputted from a mode selecting unit 107. Notethat as shown in FIG. 24B, the information indicating a motion vectorwith the highest coding efficiency may be added to a header area of amacroblock. Here, information indicating a motion vector with thehighest coding efficiency is, for example, a number identifying a blockincluding a current pixel to be referred to, and an identificationnumber given to every block. When a block is identified by theidentification number, a motion vector with the highest codingefficiency may be indicated by using only one of motion vectors used forcoding a block corresponding to an identification number, and whenmotion vectors are more than 1, a motion vector with the highest codingefficiency may be indicated by using plural motion vectors. Or, a motionvector with the highest coding efficiency may be indicated by using anidentification number given to every block to every motion vector inbi-direction (reference to the first reference picture and the secondreference picture). This selecting method makes it possible to alwaysselect a motion vector which makes coding efficiency the highest.However, since additional information showing which motion vector isselected needs to be described in a bit stream, extra amount of code forthe additional information is necessary. In addition, the thirdcalculation method is explained.

Under the third calculation method, a motion vector referring to areference picture with the smallest reference index is determined as amotion vector used in an actual direct mode. The smallest referenceindex means generally a motion vector which refers to a picture close indisplay order or a motion vector with the highest coding efficiency.Therefore, this motion vector selecting method makes it possible toimprove coding efficiency, since a motion vector used in the direct modeis generated using a motion vector which refers to a picture closest indisplay order or a motion vector with the highest coding efficiency.

Note that when all of three motion vectors refer to a same referencepicture, a median value of the three motion vectors may be used. On theother hand, when two of three motion vectors refer to a referencepicture with the smallest reference index value, for example, a same oneof the two motion vectors may be always selected. As an example,referring to FIG. 23, there are three blocks including pixels A, B and Crespectively, and when reference index values of blocks including pixelsA and B are the smallest, and a same reference picture is referred to, amotion vector in the block including the pixel A may be selected.However, when reference index values of blocks including pixels A and Care the smallest, and a same reference picture is referred to, a motionvector in a block BL1 including the pixel A located closer to a blockmay be selected.

Note that the above-mentioned median value may be a median value ofcomponents in horizontal direction and vertical direction of each motionvector, and may be a median value of value (absolute value) of eachmotion vector.

In the case as shown in FIG. 25, a median value of motion vectors may bea median value of motion vectors contained in 5 blocks: a co-locatedblock of a block BL1 in a following reference picture; blocks includingpixels A, B and C respectively; +++ and a block including a pixel Dshown in FIG. 25. As described above, when a co-located block, which isclose to a current pixel to be coded, of the block BL1 in a followingreference picture is used, a process of calculating a median value ofmotion vectors becomes easier by using a block including the pixel D inorder to make the number of blocks an odd number. Note that when pluralblocks are on a co-located area of the block BL1 in a followingreference picture, a motion compensation may be performed for the blockBL1 by using a motion vector in a block which occupies the largest areaoverlapped with the block BL1, or by dividing the block BL1corresponding to area of the plural block in the following referencepicture and a motion compensation may be performed on divided blockbasis.

An explanation will be further given using concrete examples.

As shown in FIGS. 26 and 27, when all blocks including pixels A, B and Ccontain a motion vector which refers to a picture preceding a currentpicture to be coded, any of the above-mentioned first through the thirdcalculation methods may be used.

Likewise, as shown in FIGS. 28 and 29, when all blocks including pixelsA, B and C contain a motion vector which refers to a picture following acurrent picture to be coded, any of the first through the thirdcalculation methods may be used.

Next, the case shown in FIG. 30 is explained. FIG. 30 shows the casethat each of blocks including pixels A, B and C respectively containseach of motion vectors, one refers to a picture preceding a currentpicture to be coded and another refers to a picture following a currentpicture to be coded.

According to the first calculation method, a forward motion vector usedfor motion compensation of the block BL1 is selected by a median valueof motion vectors MVAf, MVBf and MVCf, and a backward motion vector usedfor motion compensation of the block BL1 is selected by a median valueof motion vectors MVAb, MVBb and MVCb. Here, the motion vector MVAf is aforward motion vector of a block containing the pixel A, the motionvector MVAb is a backward motion vector of a block containing the pixelA, the motion vector MVBf is a forward motion vector of a blockcontaining the pixel B, the motion vector MVBb is a backward motionvector of a block containing the pixel B, the motion vector MVCf is aforward motion vector of a block containing the pixel C, the motionvector MVCb is a backward motion vector of a block containing the pixelC. Motion vectors such as the motion vector MVAf are not limited to thecase referring to a picture as shown in the figure. Same applies to afollowing explanation.

According to the second calculation method, a motion vector to be usedin the actual direct mode is determined by selecting a motion vectorwith the highest coding efficiency of forward reference motion vectorsMVAf, MVBf and MVCf, and selecting a motion vector with the highestcoding efficiency of backward reference motion vectors MVAb, MVBb andMVCb. In this case, motion compensation may be performed by only forwardreference using a motion vector with the highest coding efficiency offorward reference motion vectors MVAf, MVBf and MVCf, and bybi-prediction using a motion vector parallel with the determined motionvector. Note that, in order to achieve the highest coding efficiency, amotion compensation may be performed by selecting one block and usingforward and backward reference motion vectors of the selected blockinstead of selecting for forward and backward reference motion vectorsrespectively. In this case, since information indicating a selection canbe reduced as compared with the case that selecting informationindicating a block which contains a pixel having a forward motion vectorselected for the highest coding efficiency and a block which contains apixel having a backward motion vector selected for the highest codingefficiency, coding efficiency can be improved. The selection of theblock may be from the following: 1. A block includes a pixel having aforward reference motion vector which refers to a reference picture withthe smallest reference index value; 2. A block has the smallest valuewhich is a sum of a reference index value of a picture referred to by aforward reference motion vector of a block including each pixel and areference index value of a picture referred to by a backward referencemotion vector of a block including each pixel; 3. A block selects amedian value of reference indices of a picture referred to by a forwardreference motion vector and includes a pixel having a forward referencemotion vector with the selected median value, and a backward motionvector is included in the block; and 4. A block selects a median valueof reference indices in a picture referred to by a backward referencemotion vector and includes a pixel having a backward motion vector withthe selected median value, and a forward motion vector is included inthe block. Note that when each of backward motion vectors refers to asame picture, selecting the method 1 and the method 3 are appropriate.

According to the third calculation method, one of forward referencemotion vectors MVAf, MVBf and MVCf, which refers to a reference picturewith the smallest reference index value, is a forward reference (thefirst reference) motion vector used in the direct mode. Or, one ofbackward reference motion vectors MVAb, MVBb and MVCb, which refers to areference picture with the smallest reference index value, is a backwardreference (the second reference) motion vector used in the direct mode.Note that, in the third calculation method, the forward motion vectorreferring to the reference picture with the smallest reference index isa forward motion vector of a block BL1, and the backward motion vectorreferring to the reference picture with the smallest reference index isa backward motion vector of the block BL1, however, two motion vectorsBL1 and BL2 may be derived using one of a forward motion vector or abackward motion vector referring to a reference picture with thesmallest reference index, and motion compensation may be performed usingthe derived motion vector.

Next, the case shown in FIG. 31 is explained. FIG. 31 shows a case thatthe pixel A contains each of motion vectors, one refers to precedingpicture and another refers to a following picture, the pixel B containsonly a motion vector which refers to a preceding picture, and the pixelC contains only a motion vector which refers a following picture.

Thus, when there is a block including a pixel containing only a motionvector referring a picture uni-directionally, assuming that a motionvector referring to a picture in another direction is 0, the calculationmethod in above-mentioned FIG. 30 may be used for motion compensation.Specifically, using the first or the third calculation method in FIG.30, calculation may be performed assuming MVCf=MVBb=0. That is, in thefirst calculation method, when a forward motion vector of the block BL1is calculated, assuming that a motion vector MVCf of the pixel Creferring to a preceding picture is 0, a median value of motion vectorsMVAf, MVBf and MVCf is calculated. On the other hand, when a backwardmotion vector of the block BL1 is calculated, assuming a motion vectorMVBb, which refers to a following picture, of the pixel B is 0, a medianvalue of motion vectors MVAb, MVBb and MVCb is calculated.

According to the third calculation method, assuming that a motion vectorMVCf, which refers to a preceding picture, of the pixel C and a motionvector, which refers to a following picture, of the pixel B are 0, amotion vector, which refers to a reference picture with the smallestreference index value, of the block BL1 is calculated. For example, whena block including a pixel A refers to a picture with the first referenceindex of “0” and a block including a pixel B refers to a picture withthe first reference index of “1”, the smallest value of reference indexis “0”. Therefore, since only the motion vector MVBf, which refers to apreceding picture, of the block including the pixel refers to a picturewith the smallest first reference index, the motion vector MVBf isselected as a forward motion vector of the block BL1. Moreover, forexample, when both pixels A and C refer to a following picture with thesmallest second reference index, for example “0”, assuming a motionvector MVBb, which refers to a following picture, of a pixel B is 0, amedian value of motion vectors MVAb, MVBb and MBCb is calculated. Themotion vector resulted from the calculation is set as a forward motionvector of the block BL1.

Next, a case shown in FIG. 32 is explained. FIG. 32 shows a case whenthe pixel A contains each of motion vectors; one refers to precedingpicture and another refers to a following picture. The pixel B containsonly a motion vector which refers to a preceding picture and the pixel Cdoes not contain a motion vector and is intra-picture coded.

When a block including a current pixel C to be coded is intra-picturecoded, assuming that motion vectors which refer to pictures precedingand following the block are both 0, the calculation method inabove-mentioned FIG. 30 may be used for motion compensation.Specifically, calculation may be performed assuming MVCf=MVCb=0. Notethat, in FIG. 30, MVBb is 0.

Lastly, a case shown in FIG. 33 is explained. FIG. 33 shows a case thata pixel C is coded by the direct mode.

When blocks including a current pixel referred to contain a block codedby the direct mode, motion compensation of the block BL1 may beperformed using a motion vector used for coding a block coded by thedirect mode and using the calculation method shown in FIG. 30.

Note that it is determined which of a forward or a backward referencemotion vector is used depending on a picture to be referred to, apicture to be coded and time information included in each picture.Therefore, when a motion vector is derived after differentiating betweena forward reference and a backward reference, a motion vector of eachblock is judged if a forward reference or a backward reference from timeinformation contained in each picture.

In addition, an example of a calculation method combining theabove-mentioned calculation methods is explained. FIG. 34 is anillustration showing a procedure for determining a motion vector to beused in the direct mode. FIG. 34 is an example of a method fordetermining a motion vector using reference indices. Note that Ridx0 andRidx1 shown in FIG. 34 are reference indices explained above. FIG. 34Ashows a procedure for determining a motion vector using the firstreference index Ridx0, and FIG. 34B shows a procedure for determining amotion vector using the second reference index Ridx1. First, FIG. 34A isexplained.

In a step S3701, there are three blocks including pixels A, B and Crespectively, and the number of blocks referring to a picture using thefirst reference index Ridx0 is calculated.

When the number of blocks calculated in the step S3701 is “0”, thenumber of blocks referring to a picture using the second reference indexRidx1 is further calculated in a step S3702. When the number of blockscalculated in the step S3702 is “0”, motion compensation is performedbi-directionally for a current block to be coded assuming that a motionblock of the current block to be coded is “0” in a step S3703. On theother hand, when the number of blocks calculated in the step S3702 is“1” or more, a motion vector of a current block to be coded isdetermined in a step S3704 by the number of blocks containing the secondreference index Ridx1. For example, motion compensation of a currentblock to be coded is performed using the motion vector determined by thenumber of blocks containing the second reference index Ridx1.

When the number of blocks calculated in the step S3701 is “1”, a motionvector containing the first reference index Ridx0 is used in a stepS3705.

When the number of blocks calculated in the step S3701 is “2”, a motionvector corresponding to a median value of three motion vectors is usedin a step S3706, assuming that a block which does not contain the firstreference index Ridx0 contains a motion vector of MV=0 of the firstreference index Ridx0.

When the number of blocks calculated in the step S3701 is “3”, a motionvector corresponding to a median value of three motion vectors is usedin a step S3707. Note that motion compensation in the step S3704 may beperformed bi-directionally using one motion vector. Here, bi-directionalmotion compensation may be performed after calculating a motion vectorin the same direction as one motion vector and a motion vector in theopposite direction to one motion vector, for example, by scaling onemotion vector, or may be performed using a motion vector in the samedirection as one motion vector and a motion vector of “0”. Next, FIG.34B is explained.

The number of blocks containing the second reference index Ridx1 iscalculated in a step S3711.

When the number of blocks calculated in the step S3711 is “0”, thenumber of blocks containing the first reference index Rlxd1 is furthercalculated in a step S3712. When the number of blocks calculated in thestep S3712 is “0”, motion compensation is performed bi-directionally fora current block to be coded assuming that a motion block of the currentblock to be coded is “0” in a step S3713. On the other hand, the numberof blocks calculated in the step S3712 is “1” or more, a motion vectorof a current block to be coded is determined in a step S3714 by thenumber of blocks containing the first reference index Ridx0. Forexample, motion compensation of a current block to be coded is performedusing the motion vector determined by the number of blocks containingthe first reference index Ridx0.

When the number of blocks calculated in the step S3711 is “1”, a motionvector containing the second reference index Ridx1 is used in a stepS3715.

When the number of blocks calculated in the step S3711 is “2”, a motionvector corresponding to a median value of three motion vectors is usedin a step S3716, assuming that a block which does not contain the secondreference index Ridx1 contains a motion vector of MV=0 of the secondreference index Ridx1.

When the number of blocks calculated in the step S3711 is “3”, a motionvector corresponding to a median value of three motion vectors is usedin a step S3717. Note that motion compensation in the step S3714 may beperformed bi-directionally using one motion vector. Here, bi-directionalmotion compensation may be performed after calculating a motion vectorin the same direction as one motion vector and a motion vector in theopposite direction to one motion vector, for example, by scaling onemotion vector, or may be performed using a motion vector in the samedirection as one motion vector and a motion vector of “0”.

Note that FIGS. 34A and 34B are explained respectively, but both methodsmay be used or one of those methods may be used. However, when one ofthose methods is used, for example, when a process started from the step3704 shown in FIG. 34A is used and a process up to the step S3704 isused, a process after the step S3711 shown in FIG. 34B may be used. Whena process up to the step S3704 is used, since a process after the stepS3712 is not used, a motion vector can be determined uniquely. When bothprocesses of FIGS. 34A and 34B are used, either process may be usedfirst, or two processes may be used together. When a block neighboring acurrent block to be coded is coded in the direct mode, a reference indexreferred to by a motion vector used for coding the block coded in thedirect mode may be assumed to be contained in a block coded in thedirect mode and located in the neighbor of a current block to be coded.

Detailed explanation of a method for determining a motion vector will begiven below using concrete examples of a block. FIG. 35 is anillustration showing types of motion vectors contained in each blockreferred to by a current block BL1 to be coded. In FIG. 35A, a blockcontaining a pixel A is a block intra picture coded, and a blockcontaining a pixel B includes one motion vector and motion compensationis performed for the block using one motion vector, and a blockcontaining a pixel C is a block including two motion vectors and motioncompensation is performed bi-directionally. The block containing thepixel B contains a motion vector indicated by the second reference indexRidx1. Since the block containing the pixel A is a block to be intrapicture coded, it does not contain a motion vector. In other words, itdoes not contain a reference index, too.

In the step S3701, the number of blocks containing the first referenceindex Ridx0 is calculated. As shown in FIG. 35, since the number ofblocks containing the first reference index Ridx0 is 2, assuming that ablock which does not contain the first reference index Ridx0 contains amotion vector of MV=0 of the first reference index Ridx0, a motionvector corresponding to a median value of three motion vectors is usedin a step S3706. Bi-directional motion compensation may be performed fora current block to be coded using only the above motion vector, or maybe performed using the second reference index Ridx1 and other motionvector as shown below.

In the step S3711, the number of blocks containing the second referenceindex Ridx1 is calculated. As shown in FIG. 35, since the number ofblocks containing the second reference index Ridx1 is 1, a motion vectorcontaining the second reference index Ridx1 is used in the step S3715.

In addition, an example of another calculation method combining theabove-mentioned calculation methods is explained. FIG. 38 is anillustration showing a procedure for determining a motion vector of acurrent block to be coded using reference index values showing a picturereferred to by a motion vector contained in blocks including pixels A, Band C respectively. FIGS. 36A and 36B are illustrations showing aprocedure for determining a motion vector based on the first referenceindex Ridx0, and FIGS. 36C and 36D are illustrations showing a procedurefor determining a motion vector based on the second reference indexRidx1. FIG. 36A shows a procedure based on the first reference indexRidx0, on the other hand, FIG. 36C shows a procedure based on the secondreference index Ridx1. FIG. 36B shows a procedure based on the firstreference index Ridx0, on the other hand, FIG. 36D shows a procedurebased on the second reference index Ridx1. Hence, only FIGS. 36A and 36Bis explained below. To begin with, FIG. 36A is explained.

In a step S3801, it is judged if the smallest first reference indexRidx0 of effective first reference indices Ridx0s can be selected.

When it is possible to select the smallest first reference index Ridx0from the effective first reference indices Ridx0s, the motion vectorselected in the step S3802 is used.

When the effective first reference indices Ridx0s include the pluralsmallest reference indices Ridx0s in the step S3801, a motion vectorcontained in a block selected by priority in a step S3803 is used. Here,the priority, for example, determines a motion vector to be used formotion compensation of a current block to be coded in alphabetical orderof pixels contained in blocks.

When there is no effective first reference index Ridx0 in the stepS3801, a process that is different from the steps S3802 and S3803 isused in a step S3804. For example, a process after a step S3711explained in FIG. 34B may be used. Next, FIG. 36B is explained. Thedifferent point between FIGS. 36A and 36B is that a process in the stepsS3803 and S3804 in FIG. 36A is changed to a step S3813 in FIG. 36B.

In a step S3811, it is judged if the smallest first reference indexRidx0 of effective first reference indices Ridx0s can be selected.

When it is possible to select the smallest first reference index Ridx0from the effective first reference indices Ridx0s, the motion vectorselected in the step S3812 is used.

When there is no effective first reference index Ridx0 in the stepS3811, process that is different from S3812 is used in the step S3813.For example, a process after a step S3711 explained in FIG. 34B may beused.

Note that the above-mentioned effective first reference index Ridx0 isindicated by “O” in FIG. 35B, and is a reference index showing to have amotion vector. In FIG. 35B, the places in which “x” is written indicatesthat reference indices are not assigned. In a step S3824 in FIG. 36C andin a step S3833 in FIG. 36D, a process after the step S3701 explained inFIG. 34A may be used.

Detailed explanation of a method for determining a motion vector will begiven below using concrete examples of a block and FIG. 35.

In a step S3801, it is judged if the smallest first reference indexRidx0 of effective first reference indices Ridx0s can be selected.

In the case shown in FIG. 35, there are two effective first referenceindices Ridx0, however, when it is possible to select one smallest firstreference index Ridx0 from effective first reference indices Ridx0s inthe step S3801, a motion vector selected in the step S3802 is used.

When the effective first reference indices Ridx0s include the pluralsmallest reference indices Ridx0s in the step S3801, a motion vectorcontained in a block selected by priority in a step S3803 is used. Here,the priority, for example, determines a motion vector to be used formotion compensation of a current block to be coded in alphabetical orderof pixels contained in blocks. When blocks including pixels B and Crespectively contain the same first reference index Ridx0, the firstreference index Ridx0 of the block including the pixel B is employed bythe priority, and motion compensation is performed for a current blockBL1 to be coded using a motion vector corresponding to the firstreference index Ridx0 of the block containing the pixel B. In this case,motion compensation may be performed for the current block BL1 to becoded bi-directionally using only the determined motion vector, or maybe performed using the second reference index Ridx1 and another motionvector as shown below.

In a step S3821, it is judged if the smallest second reference indexRidx1 of effective second reference indices Ridx1s can be selected.

In the case shown in FIG. 35, since the effective second reference indexRidx1 is one, a motion vector corresponding to the second referenceindex Ridx1 of a block containing the pixel C is used in a step S3822.

Note that in the above explanation, as for the block which does notcontain reference indices, assuming that the block contains a motionvector of “0”, a median value of three motion vectors is selected,however, assuming that the block contains a motion vector of “0”, anaverage value of three vectors may be selected, or an average value ofmotion vectors of blocks containing reference indices may be selected.

Note that a motion vector used for motion compensation of a currentblock to be coded may be determined by priority different from the oneexplained above, for example, in order of pixels B-A-C, which arecontained in blocks.

Thus, through determining a motion vector used for motion compensationof a current block to be coded by using a reference index, a motionvector can be determined uniquely. Moreover, according to theabove-mentioned example, coding efficiency can be improved. In addition,since it is not necessary to judge whether a motion vector is a forwardreference or a backward reference using time information, it is possibleto simplify a process for determining a motion vector. When concerning aprediction mode for every block and a motion vector used for motioncompensation or the like, there are a lot of patterns, however, asmentioned above since a process is done by a series of flows, it isuseful.

Note that in this embodiment, the case calculating a motion vector usedin the direct mode by scaling a motion vector referred to using a timeinterval between pictures is explained, however, a calculation may beperformed by multiplying by a constant number. Here, a constant used forthe multiplication may be variable when coding or decoding is performedon plural blocks basis or on plural pictures basis. Note that acalculation method using reference indices Ridx0 and Ridx1 are not onlya method using a median value, and calculation methods may be combinedwith other calculation methods. For example, in the above-mentionedthird calculation method, when motion vectors contained in blocks havingpixels A, B and C respectively referring to a same picture of whichreference index is the smallest are plural, it is not necessary tocalculate a median value of these motion vectors, and a motion vectorobtained from averaging these motion vectors may be used as a motionvector of the block BL1 used in the direct mode. Or, for example, amotion vector of which coding efficiency is the highest may be selectedfrom plural motion vectors with the smallest reference indices.

Moreover, a forward motion vector and a backward motion vector of theblock BL1 may be calculated independently or dependently. For example, aforward motion vector and a backward motion vector may be calculatedfrom a same motion vector.

On the other hand, either a forward motion vector or a backward motionvector both obtained from the calculation may be used as a motion vectorof the block BL1.

Eighth Embodiment

In this embodiment, a reference block MB in a reference picture containsa forward (the first) motion vector referring to a reference picturestored in the long term picture buffer as the first reference pictureand a backward (the second) motion vector referring to a referencepicture stored in the short term picture buffer as the second referencepicture.

FIG. 37 is an illustration showing a bi-prediction in the direct modewhen only one reference picture is stored in the long term picturebuffer.

The different point of the eighth embodiment from above-mentioned pluralembodiments is that a forward motion vector MV21 of a block MB2 in areference picture refers to a reference picture stored in the long termpicture buffer.

The short term picture buffer is a buffer for storing reference picturestemporarily, and, for example, pictures are stored in order in whichpictures are stored in a buffer (that is, coding/decoding order). Whenpictures are newly stored in the buffer there is not enough storagecapacity, pictures are deleted from a picture stored most previously inthe buffer.

In the long term picture buffer, pictures are not always stored in thelong term picture buffer in time order as the short term picture buffer.For example, as an order of storing pictures, time order of pictures maybe corresponded, and order of address in a buffer in which pictures arestored may be corresponded. Therefore, it is impossible to scale amotion vector MV21 referring to a picture stored in the long termpicture buffer based on a time interval.

A long term picture buffer is not for storing a reference picturetemporarily as the short term picture buffer, but for storing areference picture continuously. Therefore, a time interval correspondingto a motion vector stored in the long term picture buffer is much widerthan a time interval corresponding to a motion vector stored in theshort term picture buffer.

In FIG. 37, a boundary between the long term picture buffer and theshort term picture buffer is indicated by a dotted vertical line asshown in the figure, and information about pictures on the left side ofthe dotted vertical line is stored in the long term picture buffer, andinformation about pictures on the right side of the dotted vertical lineis stored in short term picture buffer. Here, a block MB1 in a pictureP23 is a current block. A block MB2 is a co-located reference block ofthe block MB1 in a picture P24. A forward motion vector MV21 of theblock MB2 in the reference picture P24 is the first motion vectorreferring to a picture P21 stored in the long term picture buffer as thefirst reference picture, and a backward motion vector MV25 of the blockMB2 in the reference picture P24 is the second motion vector referringto a picture P25 stored in the short term picture buffer as the secondreference picture.

As mentioned above, a time interval TR21 between the pictures P21 andP24 is corresponded to a forward motion vector MV21 referring to apicture stored in the long term picture buffer, a time interval TR25between the pictures P24 and P25 is corresponded to a backward motionvector MV25 referring to a picture stored in the short term picturebuffer, and the time interval TR21 between the pictures P21 and P24 canbecome much wider than the time interval TR25 between the pictures P24and P25 or can be undefined.

Therefore, a motion vector of the block MB1 in the current picture P23is not calculated by scaling a motion vector of the block MB2 in thereference picture P24 as previous embodiments, but the motion vector iscalculated using a following method.MV21=MV21′MV24′=0

The upper equation shows that the first motion vector MV21 stored in thelong term picture buffer is used directly as the first motion vectorMV21′ in a current picture.

The lower equation shows that since the second motion vector MV24′,which refers to the picture 24 stored in the short term picture buffer,of the block MB1 in the current picture for the picture P23 is enoughsmaller than the first motion vector MV21′, MV24′ is negligible. Thesecond motion vector MV24′ is treated as “0”.

As described above, a reference block MB contains one motion vectorreferring to a reference picture stored in the long term picture bufferas the first reference picture; and one motion vector referring to areference picture stored in the short term picture buffer as the secondreference picture. In this case, bi-prediction is performed using themotion vector stored in the long term picture buffer out of motionvectors of the block in the reference picture directly as a motionvector of a block in a current picture.

Note that a reference picture stored in the long term picture buffer maybe either the first reference picture or the second reference picture,and a motion vector MV21 referring to a reference picture stored in thelong term picture buffer may be a backward motion vector. Moreover, whenthe second reference picture is stored in the long term picture bufferand the first reference picture is stored in the short term picturebuffer, a motion vector in a current picture is calculated by scaling amotion vector referring to the first reference picture.

This makes it possible to perform bi-prediction without using time whichis considerably large in the long term picture buffer or undefined.

Note that bi-prediction may be performed not directly using a motionvector referred to but using a motion vector by multiplying by aconstant number.

In addition, a constant used for the multiplication may be variable whencoding or decoding is performed on plural blocks basis or on pluralpictures basis.

Ninth Embodiment

In this embodiment, bi-prediction in the direct mode is shown. In thiscase, a reference block MB in a reference picture contains two forwardmotion vectors referring to a reference picture stored in a long termpicture buffer.

FIG. 38 is an illustration showing bi-prediction in the direct mode whena reference block MB in a reference picture contains two forward motionvectors referring to a reference picture stored in the long term picturebuffer.

The different point of the ninth embodiment from the eighth embodimentis that both motion vectors MV21 and MV22 of a block MB2 in a referencepicture refer to a picture stored in the long term picture buffer.

In FIG. 38, a boundary between the long term picture buffer and theshort term picture buffer is indicated by a dotted vertical line asshown in the figure, and information about pictures on the left side ofthe dotted vertical line is stored in the long term picture buffer andinformation about pictures on the right side of the dotted vertical lineis stored in short term picture buffer. Motion vectors MV21 and MV22 ofthe block MB2 in a reference picture P24 both refer to a picture storedin the long term picture buffer. The motion vector MV21 corresponds to areference picture P21, and the motion vector MV22 corresponds to areference picture P22.

A time interval TR22 between the pictures P22 and P24 can become muchwider than the time interval TR25 between the pictures P24 and P25 orcan be undefined corresponding to the motion vector MV22 referring tothe picture P22 stored in the long term picture buffer.

In FIG. 38, pictures are stored in order of pictures P22-P21 in thatorder in the long term picture buffer. Here the picture P21 correspondsto a motion vector MV21 and the picture P22 corresponds to a motionvector MV22. In FIG. 38, a motion vector of a block MB1 in a currentpicture is calculated as follows.MV22′=MV22MV24′=0

The upper equation shows that a motion vector MV22 referring to apicture P22 to which the smallest order is assigned is used directly asa motion vector MV22′ of the block MB1 in a current picture P23.

The lower equation shows that since the backward motion vector MV24′ ofthe block MB1 in the current picture P23 stored in the short termpicture buffer is enough smaller than the motion vector MV21′, MV24′ isnegligible. The backward motion vector MV24′ is treated as “0”.

As described above, by directly using a motion vector referring to apicture to which the smallest order is assigned out of motion vectors ofa block in a reference picture stored in the long term picture buffer,bi-prediction can be made without using time which is considerably largein the long term picture buffer or undefined.

Note that bi-prediction may be made not directly using a motion vectorreferred to but using a motion vector by multiplying by a constantnumber.

In addition, a constant used for the multiplication may be variable whencoding or decoding is performed on plural blocks basis or on pluralpictures basis.

Furthermore, when motion vectors MV21 and MV22 of the block MB2 in areference picture both refer to a picture stored in the long termpicture buffer, a motion vector referring to the first reference picturemay be selected. For example, when MV21 is a motion vector referring tothe first reference picture and MV22 is a motion vector referring to thesecond reference picture, the motion vector MV21 referring to a pictureP21 and a motion vector “0” referring to a picture P24 are used asmotion vectors of a block MB1.

Tenth Embodiment

In this embodiment, a calculation method of a motion vector in thedirect mode shown in the fifth embodiment through the ninth embodimentis explained. This calculation method of a motion vector is applied toeither of coding and decoding a picture. Here, a current block to becoded or decoded is called a current block MB. A co-located block of thecurrent block MB in a reference picture is called a reference block.

FIG. 39 is an illustration showing a process flow of a motion vectorcalculation method of this embodiment.

First, it is judged if a reference block MB in a backward referencepicture referred by a current block MB contains a motion vector (stepS1). If the reference block MB does not contain a motion vector (No instep S1), bi-prediction is performed assuming that a motion vector is“0” (step S2) and a process for a motion vector calculation iscompleted.

If the reference block MB contains a motion vector (Yes in step S1), itis judged if the reference block contains a forward motion vector (stepS3).

If the reference block does not contain a forward motion vector (No instep S3), since the reference block MB contains only a backward motionvector, the number of backward motion vectors is judged (step S14). Whenthe number of backward motion vectors of the reference block MB is “2”,bi-prediction is performed using two backward motion vectors scaledbased on one of the calculation method mentioned in FIGS. 17, 18, 19 and20.

On the other hand, when the number of backward motion vectors of thereference block MB is “1”, the only backward motion vector contained inthe reference block MB is scaled and motion compensation is performedusing the scaled backward motion vector (step S16). After completing thebi-prediction in the step S15 or S16, a process of the motion vectorcalculation method is completed.

On the other hand, if the reference block MB contains a forward motionvector (Yes in step S3), the number of forward motion vectors of thereference block MB is judged (step S4).

When the number of forward motion vectors of the reference block MB is“1”, it is judged if a reference picture corresponding to the forwardmotion vector of the reference block MB is stored in the long termpicture buffer or the short term picture buffer (step S5).

When the reference picture corresponding to the forward motion vector ofthe reference block MB is stored in the short term picture buffer, theforward motion vector of the reference block MB is scaled andbi-prediction is performed using the scaled forward motion vector (stepS6).

When the reference picture corresponding to the forward motion vector ofthe reference block MB is stored in the long term picture buffer,bi-prediction is performed based on the motion vector calculation methodshown in FIG. 37 assuming that a backward motion vector is 0 and usingthe forward motion vector of the reference block MB directly withoutscaling (step S7). After completing the bi-prediction in the step S6 orS7, a process of the motion vector calculation method is completed.

When the number of forward motion vectors of the reference block MB is“2”, the number of forward motion vectors corresponding to a referencepicture stored in the long term picture buffer is judged (step S8).

When the number of forward motion vectors corresponding to a referencepicture stored in the long term picture buffer is “0” in the step S8, amotion vector which is temporally close to a current picture containingthe current block MB is scaled and bi-prediction is performed using thescaled forward motion vector based on the motion vector calculationmethod shown in FIG. 14 (step S9).

When the number of forward motion vectors corresponding to a referencepicture stored in the long term picture buffer is “1” in the step S8, amotion vector in a picture stored in the short term picture buffer isscaled and bi-prediction is performed using the scaled motion vector(step S10).

When the number of forward motion vectors corresponding to a referencepicture stored in the long term picture buffer is “2” in the step S8, itis judged if a same picture in the long term picture buffer is referredto by both of two forward motion vectors (step S11). If the same picturein the long term picture buffer is referred to by both of two forwardmotion vectors (Yes in step S11), bi-prediction is performed using amotion vector previously coded or decoded in the picture referred to bytwo forward motion vectors in the long term picture buffer based on themotion vector calculation method shown in FIG. 13 (step S12).

If a same picture in the long term picture buffer is not referred to byboth of two forward motion vectors (No in step S11), bi-prediction isperformed using a forward motion vector corresponding to a picture towhich a small order is assigned in the long term picture buffer (stepS13). In the long term picture buffer, since reference pictures arestored regardless of actual time of pictures, a forward motion vector tobe used for bi-prediction is selected in concordance with an orderassigned to each reference picture. There is a case that the order ofreference pictures stored in the long term picture buffer coincides withtime of pictures, however, it may be merely coincided with an address inthe buffer. In other words, the order of pictures stored in the longterm picture buffer does not necessarily coincide with time of pictures.After completing bi-prediction in steps S12 and S13, process of themotion vector calculation method is completed.

Eleventh Embodiment

Detailed explanation of the eleventh embodiment according to the presentinvention will be given below using illustrations.

FIG. 40 is a block diagram showing a configuration of a moving picturecoding apparatus 1100 according to the eleventh embodiment of thepresent invention. The moving picture coding apparatus 1100 is anapparatus which can code moving pictures by applying a spatialprediction method in the direct mode even if a block coded in a fieldstructure and a block coded in a frame structure are mixed, and includesa frame memory 1101, a difference calculating unit 1102, a predictivedifference coding unit 1103, a bit stream generating unit 1104, apredictive difference decoding unit 1105, an add operating unit 1106, aframe memory 1107, a motion vector detecting unit 1108, a mode selectingunit 1109, a coding control unit 110, a switch 1111, a switch 1112, aswitch 1113, a switch 1114, a switch 1115 and a motion vector storingunit 1116.

The frame memory 1101 is a picture memory storing inputted pictures onpicture basis. The difference calculating unit 1102 calculatesprediction error, which is difference between an inputted picture fromthe frame memory 1101 and a reference picture obtained from a decodedpicture based on a motion vector, and outputs it. The predictiondifference coding unit 1103 performs frequency conversion for theprediction error obtained in the difference calculating unit 1102,quantizes and outputs it. The bit stream generating unit 1104 convertsinto a format of output coded bit stream after performing variablelength coding of the coded result from the predictive difference codingunit 1103, and generates a bit stream adding additional information suchas header information in which related information on the codedprediction error is described. The predictive difference decoding unit1105 performs variable length coding and inverse quantization of thecoded result from the predictive difference coding unit 1103, and afterthat performs inverse frequency conversion such as IDCT conversion afterperforming, and decodes the coded result to output predictive residual.The add operating unit 1106 adds a predictive residual as a decodedresult, to the above-mentioned reference picture, and outputs areference picture showing a same picture as an inputted picture by codedand decoded picture data. The frame memory 1107 is a picture memorystoring reference pictures on picture basis.

The motion vector detecting unit 1108 derives a motion vector for everycoding of a current frame to be coded. The mode selecting unit 1109selects if calculation of a motion vector is performed in the directmode or in other mode. The coding control unit 1110 reorders inputtedpictures stored in the frame memory 1101 in input order to coding order.Additionally, the coding control unit 1110 judges which of a fieldstructure or a frame structure is used for coding for everypredetermined size of a current frame to be coded. Here, thepredetermined size is a size of two macroblocks (for example, 16(horizontal)×16 (vertical) pixels) combined vertically (hereinafter,macroblock pair). If a field structure is used for coding, a pixel valueis read every other horizontal scanning line corresponding to interlacefrom the frame memory 1101, if a frame basis is used for coding, eachpixel value in inputted picture is read sequentially from the framememory 1101 and each read pixel value is placed on the memory in orderto configure a current macroblock pair to be coded corresponding to thefield structure or the frame structure. The motion vector storing unit1116 stores a motion vector of a coded macroblock and reference indicesof frames referred to by the motion vector. Reference indices are storedfor each macroblock of coded macro block pairs.

Next, an operation of the moving picture coding apparatus configured asabove is explained. Pictures to be inputted are inputted into the framememory 1101 on picture basis in time order. FIG. 41A is an illustrationshowing an order of frames inputted into the moving picture codingapparatus 100 on picture basis in time order. FIG. 41B is anillustration showing an order of pictures reordering the order ofpictures shown in FIG. 41A to coding order. In FIG. 41A, a verticallines indicate pictures, and the number indicated on the lower rightside each picture shows the picture types (I, P and B) with the firstalphabet letters and the picture numbers in time order with followingnumbers. FIG. 42 is an illustration showing a structure of a referenceframe list 300 to explain the eleventh embodiment. Each picture inputtedinto the frame memory 1101 is reordered in coding order by the codingcontrol unit 1110. Pictures are reordered in coding order based onreferential relation of an inter picture prediction coding, and in thecoding order, a picture used as a reference picture is coded previouslyto a picture referring to a picture.

For example, it is assumed that a P picture uses one of preceding andneighboring three I or P pictures as a reference picture. On the otherhand, it is assumed that a B picture uses one of preceding andneighboring three I or P pictures and one of following and neighboring Ior P picture as a reference picture. Specifically, a picture P7 which isinputted after pictures B5 and B6 in FIG. 41A is reordered and placedbefore pictures B5 and B6 since the picture P7 is referred to bypictures B5 and B6. Likewise, a picture P10 inputted after pictures B8and B9 is reordered and placed before pictures B8 and B9, and a pictureP13 inputted after pictures B11 and B12 is reordered and placed beforepictures B11 and B12. Hence, the result of reordering an order ofpictures shown in FIG. 41A is as shown in FIG. 41B.

It is assumed that each picture reordered by the frame memory 1101 isread on macroblock pair basis, and each macroblock pair is 16(horizontal)×16 (vertical) pixels in size. Here a macroblock paircombines two macroblocks vertically. Therefore, a macroblock pair is 16(horizontal)×32 (vertical) pixels in size. A process of coding a pictureB11 is explained below. Note that in this embodiment it is assumed thatthe coding control unit 1110 controls reference indices, that is, areference frame list.

Since the picture B11 is a B picture, inter picture prediction coding isperformed using bi-directional reference. It is assumed that the pictureB11 uses two of preceding pictures P10, P7, P4, and a following pictureP13 as a reference picture. Additionally, it is assumed that selectionof two pictures from these four pictures can be specified on macroblockbasis. Here, it is assumed that reference indices are assigned using amethod of initial condition. In other words, a reference frame list 300during coding the picture B11 is as shown in FIG. 42. Regarding areference picture in this case, the first reference picture is specifiedby the first reference index in FIG. 42 and the second reference pictureis specified by the second reference index.

In a process for the picture B11, it is assumed that the coding controlunit 1110 controls the switch 1113 to be on, the switches 1114 and 1115to be off. Therefore, a macroblock pair in the picture B11 read from theframe memory 1101 is inputted into the motion vector detecting unit1108, the mode selecting unit 109 and difference calculating unit 1102.In the motion vector detecting unit 108, by using decoded data ofpictures P10, P7 and P4 stored in the frame memory 1107 as a referencepicture, the first motion vector and the second motion vector of eachmacroblock contained in a macroblock pair is derived. In the modeselecting unit 1109, a coding mode for a macroblock pair is determinedusing motion vectors derived in the motion vector detecting unit 1108.Here, it is assumed that coding mode for a B picture may be selectedfrom, for example, intra picture coding, inter picture prediction codingusing uni-directional motion vector, inter picture prediction codingusing bi-directional motion vector and a direct mode. When coding modesother than the direct mode are selected, it is determined which one of aframe structure or a field structure is used for coding a macroblockpair at the same time.

Here, a motion vector calculation method using a spatial predictingmethod in the direct mode is explained. FIG. 43A is a flow chart showingan example of a motion vector calculation procedure using a spatialpredicting method in the direct mode when a macroblock pair to be codedin a field structure and a macroblock pair to be coded in a framestructure are mixed.

FIG. 43B is an illustration showing an example of a location ofneighboring macroblock pairs to which the present invention is appliedwhen a current macroblock pair to be coded is coded in a framestructure. FIG. 43C is an illustration showing an example of location ofneighboring macroblock pairs to which the present invention is appliedwhen a current macroblock pair to be coded is coded in a fieldstructure. A macroblock pair diagonally shaded in FIGS. 43B and 43C is acurrent macroblock pair to be coded.

When a current macroblock pair to be coded is coded using a spatialprediction in the direct mode, three coded macroblock pairs in theneighbor of the current macroblock pair to be coded are selected. Inthis case, the current macroblock pair may be coded in either of a fieldstructure and a frame structure. Therefore, the coding control unit 1110first determines which one of a field structure or a frame structure isused for coding a current macroblock pair to be coded. For example, whenthe number of neighboring macroblock pairs coded in a field structure isgreat, a current macroblock pair is coded in a field structure, and ifthe number of neighboring macroblock pairs coded in a frame structure isgreat, a current macroblock pair is coded in a field structure. Thus, bydetermining which one of a field structure or a frame structure is usedfor coding a current macroblock pair to be coded by using information onneighboring blocks, it is not necessary to describe information showingwhich structure is used for coding a current macroblock pair to be codedin a bit stream. Additionally, since a structure is predicted fromneighboring macroblock pairs, it is possible to select an adequatestructure.

Next, the motion vector detecting unit 1108 calculates a motion vectorof a current macroblock pair to be coded according to determination ofthe coding control 1110. First, the motion vector detecting unit 1108checks which one of a field structure or a frame structure is determinedto be used for coding by the control unit 110 (S301), and when the framestructure is determined to be used for coding, a motion vector of acurrent macroblock pair to be coded is derived using the frame structure(S302), and when the field structure is determined to be used forcoding, a motion vector of the current macroblock pair to be coded isderived using the field structure (S303).

FIG. 44 is an illustration showing a data configuration of a macroblockpair when coding is performed using a frame structure, and a dataconfiguration of a macroblock pair when coding is performed using afield structure. In FIG. 44, a white circle indicates a pixel onodd-numbered horizontal scanning lines, and a black circle shaded with ahatch pattern of oblique lines indicates a pixel on even-numberedhorizontal scanning lines. When a macroblock pair is cut from each frameshowing an inputted picture, pixels on odd-numbered horizontal scanninglines and pixels on even-numbered horizontal scanning lines are placedalternately in a vertical direction as shown in FIG. 44. When theabove-mentioned macroblock pair is coded in the frame structure, aprocess is performed every macroblock MB1 and every macroblock MB2 forthe macroblock pair, and a motion vector is calculated for each ofmacroblocks MB1 and MB2 forming a macroblock pair. When the macroblockpair is coded in the field structure, the macroblock pair is dividedinto a macroblocks TF and BF. Here, the macroblock TF indicates a topfield and the macroblock BF indicates a bottom field when interlacing amacroblock pair in a horizontal scanning line direction, and the twofields form a macroblock pair. Then one motion vector is calculated forthe two fields respectively.

Based on such a macroblock pair, the case that a current macroblock pairto be coded is coded in a frame structure is explained as shown in FIG.43B. FIG. 45 is a flow chart showing a detailed processing procedure ina step S302 shown in FIG. 43. Note that in FIG. 45 a macroblock pair isindicated as MBP, and a macroblock is indicated as MB.

The mode selecting unit 1109 first calculates a motion vector of amacroblock MB1 (an upper macroblock), which is one of macroblocksforming a current macroblock pair to be coded, using a spatialprediction in the direct mode. First, the mode selecting unit 1109calculates the smallest value of indices in pictures referred to byneighboring macroblock pairs for the first and the second indicesrespectively (S501). In this case, however, when a neighboringmacroblock pair is coded in the frame structure, the value is determinedusing only a macroblock adjacent to a current macroblock to be coded.Next, it is checked if neighboring macroblock pairs are coded in thefield structure (S502), and if coding is performed using the fieldstructure, it is further checked the number of fields to which thesmallest index are assigned in fields referred to by two macroblocksforming the neighboring macroblock pairs from a reference frame list inFIG. 42 (S503).

When the check result of the step S503 shows that the smallest index isassigned to either field referred to by the two macroblocks (that is,fields to which the same index is assigned), an average value of motionvectors of two macroblocks is calculated and made to be a motion vectorof the neighboring macroblock pair. This is since, when consideringbased on a interlace structure, two macroblocks of neighboringmacroblock pairs with the field structure are adjacent to a currentmacroblock to be coded with the frame structure.

When the check result of the step S503 shows that the smallest index isassigned to only a field referred to by one macroblock; a motion vectorof the macroblock is determined as a motion vector of the neighboringmacroblock pair (S504A). When the smallest index is assigned to none offields referred to, a motion vector of the neighboring macroblock pairis assumed to be “0” (S505).

In above cases, from motion vectors of the neighboring macroblocks, byusing only motion vectors referring to fields to which the smallestindex is assigned, it is possible to select a motion vector with highercoding efficiency. A process in a step S505 shows that there is noadequate motion vector for prediction.

When the check result of the step S502 shows the neighboring macroblockpairs are coded in the frame structure, among the neighboring macroblockpairs, a motion vector of a macroblock adjacent to a current macroblockto be coded is determined as a motion vector of the neighboringmacroblock pair (S506).

The mode selecting unit 1109 repeats processes from above steps S501 toS506 for selected three neighboring macroblock pairs. As a result, amotion vector is calculated for each of three neighboring macroblockpairs as for one macroblock of a current macroblock pair to be coded,for example, a macroblock MB1.

Next, the mode selecting unit 1109 checks if the number of neighboringmacroblock pairs referring to a frame with the smallest index or a fieldof the frame among three neighboring macroblock pairs is 1 (S507).

In this case, the mode selecting unit 1109 unifies reference indices ofthree neighboring macroblock pairs to a reference frame index or areference field index, and compares them. In a reference frame listshown in FIG. 42, reference indices are merely assigned to every frame,however, since relation between the reference frame indices andreference field indices to which indices are assigned every field areconstant, it is possible to convert one of reference frame list or areference field list into another reference indices by calculation.

FIG. 46 is an indicator chart showing a relation between reference fieldindices and reference frame indices.

As shown in FIG. 46, there are several frames indicated by the firstfield f1 and the second field f2 in chronological order in a referencefield list, and reference frame indices such as 0, 1, and 2 are assignedto each frame based on frames including a current block to be coded(frames shown in FIG. 46). In addition, reference field indices such as0, 1 and 2 are assigned to the first field f1 and the second field f2 ofeach frame based on the first field f1 of a frame including a currentblock to be coded (when the first field is a current field to be coded).Note that the reference field indices are assigned from the first fieldf1 and the second field f2 of a frame close to a current field to becoded. Here, if the current block to be coded is the first field f1, theindices are assigned giving priority to the first field f1, and if thecurrent block to be coded is the second field f2, the indices areassigned giving priority to the second field f2.

For example, when a neighboring macroblock coded in the frame structurerefers to a frame with a reference frame index “1” and a neighboringmacroblock coded in the field structure refers to the first field f1with a reference field index “2”, the above-mentioned neighboringmacroblocks are treated as they refer to a same picture. In other words,when a precondition that a reference frame index referred to by oneneighboring macroblock is equal to half the reference field index (rounddown after decimal point) assigned to a reference field of anotherneighboring macroblock is satisfied, the neighboring macroblocks aretreated as they refer to a same picture.

For example, when a current block to be coded included in the firstfield f1 indicated by Δ in FIG. 46 refers to the first field f1 with thereference field index “2”, and a neighboring macroblock with the framestructure refers to a frame with the reference frame index “1”, theabove-mentioned neighboring blocks are treated as they refer to a samepicture since the above-mentioned precondition is satisfied. On theother hand, when a neighboring macroblock refers to the first field witha reference field index “2” and other neighboring macroblock refers to aframe with a reference frame index “3”, the neighboring blocks aretreated as they do not refer to a same picture since the precondition isnot satisfied.

As mentioned above, if the check result of the step S507 shows thenumber is 1, a motion vector of a neighboring macroblock pair referringto a field a frame with the smallest index or a field in the frame isdetermined as a motion vector of a current macroblock to be coded(S508). If the check result of the step S507 shows the number is not 1,it is further checked if the number of neighboring macroblock pairs ofthree neighboring macroblock pairs referring to a frame with thesmallest index or a field in the frame is 2 or more (S509). Then if thenumber is 2 or more, assuming that a motion vector of neighboringmacroblock pairs not referring to a frame with the smallest index or afield in the frame is “0” (S510), a median value of three motion vectorsof the neighboring macroblock pairs is determined as a motion vector ofa current macroblock to be coded (S511). If the check result of the stepS509 is less than 2, since the number of the neighboring macroblockpairs referring to the frame with the smallest index or the field in theframe is “0”, a motion vector of a current macroblock to be coded isdetermined as “0” (S512).

As a result of the above process, one motion vector MV1 can be obtainedas a calculation result for a macroblock forming a current macroblockpair to be coded, for example, MB1. The mode selecting unit 109 performsthe above process for a motion vector with the second reference index,and performs motion compensation by bi-prediction using the two obtainedmotion vectors. However, when none of neighboring macroblocks containsthe first or the second motion vector, motion compensation is performednot using a motion vector in the direction indicated by a motion vectornot contained in the neighboring macroblocks but using a motion vectorin uni-direction. Moreover, the same process is repeated for the othermacroblock in the current macroblock pair to be coded, for example, MB2.As a result, it is equal to perform motion compensation in the directmode for each of two macroblocks in a current macroblock pair to becoded.

Next, the case that a current macroblock pair to be coded is coded inthe field structure as shown in FIG. 43C is explained. FIG. 47 is a flowchart showing a detailed processing procedure in a step S303 shown inFIG. 43. The mode selecting unit 1109 calculates one motion vector MVtusing a spatial prediction in the direct mode for a macroblock forming acurrent macroblock pair to be coded, for example, a macroblock TFcorresponding to a top field of the macroblock pair. First, the modeselecting unit 1109 calculates the smallest value of indices in picturesreferred to by neighboring macroblock pairs (S601). However, when themacroblock pairs are processed by the field structure, only a macroblockof a field (a top field or a bottom field) same as the currentmacroblock to be coded is considered. Next, it is checked if theneighboring macroblock pairs are coded by the frame structure (S602),and if coding is performed using the frame structure, it is furtherjudged if frames referred to by two macroblocks in the neighboringmacroblock pair are frames with the smallest index based on the indexvalue assigned to each frame by a reference frame list 300 (S603).

If the check result of the step S603 shows that the smallest index isassigned to either of the frames referred to by the two macroblocks, anaverage value of motion vectors of the two macroblocks is calculated,and the calculation result is determined as a motion vector of theneighboring macroblock pair (S604). If the check result of the step S603shows that one or both of the frames referred to are not frames with thesmallest index, it is further checked if a frame referred to by eitherof macroblocks contains the smallest index (S605). If the check resultshows that the smallest index is assigned to a frame referred to by oneof macroblocks, a motion vector of the macroblock is determined as amotion vector of the neighboring macroblock pair (S606). On the otherhand, if the check result of the step S605 shows that none ofmacroblocks refers to a frame with the smallest index, a motion vectorof the neighboring macroblock pair is determined as “0” (S607). In abovecases, from motion vectors of the neighboring macroblocks, by using onlymotion vectors referring to frames to which the smallest index isassigned, it is possible to select a motion vector with higher codingefficiency. A process in a step S607 shows that there is no adequatemotion vector for prediction.

When the check result of the step S602 shows the neighboring macroblockpairs are coded in the field structure, in the neighboring macroblockpair, motion vectors of the whole neighboring macroblock pair isdetermined as a motion vector of the macroblock pair corresponding to acurrent macroblock in a current macroblock pair to be coded (S608). Themode selecting unit 109 repeats processes from above steps S601 to S608for selected three neighboring macroblock pairs. As a result, it isequal to obtain a motion vector for three neighboring macroblock pairsrespectively as for a macroblock of the current macroblock pair to becoded, for example, a macroblock TF.

Next, the motion vector detecting unit 108 checks if the number ofneighboring macroblock pairs referring to a frame with the smallestindex among three neighboring macroblock pairs is 1 (S609). If it is 1,a motion vector of a neighboring macroblock pair referring to a framewith the smallest index is determined as a motion vector of the currentmacroblock to be coded (S610). If the check result of the step S609shows the number is not 1, it is further checked if the number ofneighboring macroblock pairs referring to a frame with the smallestindex among three neighboring macroblock pairs is two or more (S611).Then if the number is two or more, assuming that a motion vector ofneighboring macroblock pairs not referring to a frame with the smallestindex is “0” (S612), a median value of three motion vectors ofneighboring macroblock pairs is determined as a motion vector of thecurrent macroblock to be coded (S613). If the check result of the stepS611 is less than 2, since the number of neighboring macroblock pairsreferring to a frame with the smallest index is “0”, a motion vector ofthe current macroblock to be coded is determined as “0” (S614).

As a result of the above process, a motion vector MVt can be obtained asa calculation result for a macroblock forming a current macroblock pairto be coded, for example, a macroblock TF corresponding to a top field.The mode selecting unit 109 repeats the above process also for thesecond motion vector (corresponding to the second reference index). Asfor a macroblock TF, two motion vectors can be obtained by aboveprocess, and motion compensation is performed using the two motionvectors. However, when none of neighboring macroblocks contains thefirst or the second motion vector, motion compensation is performed notusing a motion vector in the direction indicated by a motion vector notcontained in the neighboring macroblocks but using a motion vector inuni-direction. This is because when a neighboring macroblock pair refersto only uni-directionally, it is conceivable that coding efficiencybecomes higher when a neighboring macroblock pair also refers to onlyuni-direction.

Moreover, the same process is repeated for another macroblock in thecurrent macroblock pair to be coded, for example, BF corresponding to abottom field. As a result, it is equal to perform motion compensation inthe direct mode for each of two macroblocks in the current macroblockpair to be coded, for example, the macroblocks TF and BF.

Note that in the above cases, when a coding structure for a currentmacroblock pair to be coded and a coding structure for a neighboringmacroblock pair are different, a calculation is performed by a processsuch as calculating an average value of motion vectors of twomacroblocks in the neighboring macroblock pair, however, the presentinvention is not limited to the above cases. For example, only when acoding structure for a current macroblock pair to be coded and aneighboring macroblock pair are the same, a motion vector of theneighboring macroblock pair may be used, and when a coding structure fora current macroblock pair to be coded and a neighboring macroblock pairare different, a motion vector of the neighboring macroblock pair ofwhich coding structure is different is not used. Specifically, 1. When acurrent macroblock pair to be coded is coded in the frame structure,only a motion vector of a neighboring macroblock pair coded in the framestructure is used. In this case, when none of motion vectors of theneighboring macroblock pair coded in the frame structure refers to aframe with the smallest index, a motion vector of the current macroblockpair to be coded is determined as “0”. When a neighboring macroblockpair is coded in the field structure, a motion vector of the neighboringmacroblock pair is determined as “0”.

Next, 2. When a current macroblock pair to be coded is coded in thefield structure, only a motion vector of a neighboring macroblock paircoded in a field structure is used. In this case, when none of motionvectors of the neighboring macroblock pair coded in the field structurerefers to a frame with the smallest index, a motion vector of thecurrent macroblock pair to be coded is determined as “0”. When aneighboring macroblock pair is coded in the frame structure, a motionvector of the neighboring macroblock pair is determined as “0”. Thus,after calculating a motion vector of each neighboring macroblock pair,3. When the number of motion vectors obtained by referring to a framewith the smallest index or a field in the frame among these motionvectors is only one, the motion vector is determined as a motion vectorof a current macroblock pair in the direct mode, and if the number isnot 1, a median value of three motion vectors is determined as a motionvector of the current macroblock pair in the direct mode.

Additionally, in the above cases, which one of a field structure or aframe structure is used for coding a current macroblock pair to be codedis determined based on the majority of a coding structure of codedneighboring macroblock pairs, however, the present invention is notlimited to the above case. A coding structure may be fixed, for example,a frame structure is always used for coding in the direct mode, or afield structure is always used for coding in the direct mode. In thiscase, for example, when the field structure and the frame structure areswitched for an every current frame to be coded, it may be described ina header of whole bit stream or in a frame header of every frame.Switching is performed, for example, on sequence basis, GOP basis,picture basis, and slice basis, and in this case, it may be described ina corresponding header of a bit stream or the like. Needless to say,even in the above cases, only when coding structures used for a currentmacroblock pair to be coded and a neighboring macroblock pair are thesame, a motion vector of the current macroblock pair to be coded in thedirect mode can be calculated by a method using a motion vector of theneighboring macroblock pair. In addition, when transmitting in such aspackets, a header part and a data part may be separated and transmittedrespectively. In this case, the header part and the data part are neverincluded in one bit stream. However, as for packets, although atransmission order can be more or less out of sequence, a header partcorresponding to a corresponding data part is just transmitted in otherpacket, and there is no difference even if it is not included in one bitstream. Thus, through fixing which one of the frame structure or thefield structure is used, a process for determining a coding structure byusing information of neighbor becomes unnecessary, and a process can besimplified.

In addition, in the direct mode, after processing a current macroblockpair using both the frame structure and the field structure, a codingstructure with higher coding efficiency may be selected. In this case,it may be described which one of the frame structure and the fieldstructure is selected in a header part of a macroblock pair undercoding. Needless to say, even in the above cases, only when codingstructures used for a current macroblock pair to be coded and aneighboring macroblock pair are the same, a motion vector of the currentmacroblock pair to be coded in the direct mode can be calculated by amethod using a motion vector of the neighboring macroblock pair. Byusing such a method, information showing which one of the framestructure or the field structure is used becomes necessary in a bitstream, however, it is possible to reduce residual data for motioncompensation and coding efficiency can be improved.

In the above explanation, the case that motion compensation is performedfor neighboring macroblock pairs on macroblock size basis, however,motion compensation may be performed on a different size basis. In thiscase, as shown in FIGS. 48A and 48B, a motion vector containing pixelslocated on a, b and c is used as a motion vector of neighboringmacroblock pair for each macroblock of a current macroblock pair to becoded. FIG. 48A shows the case processing an upper macroblock, and FIG.48B shows the case processing a lower macroblock. When structures (aframe structure/a field structure) for a current macroblock pair andneighboring macroblock pairs are different, a process is performed usinga block including pixels located on a, b and c, and a block includingpixels located on a′, b′ and c′ as shown in FIGS. 49A and 49B. Here,locations a′, b′ and c′ are a block included in another macroblock inthe same macroblock pair corresponding to locations of pixels a, b, andc. For example, in the case of FIG. 49A, when coding structures (a framestructure/a field structure) for a current macroblock pair andneighboring macroblock pairs are different, a motion vector of a blockon the left side of an upper current macroblock to be coded isdetermined using motion vectors of BL1 and BL2. In the case of FIG. 49B,when coding structures (a frame structure/a field structure) for acurrent macroblock pair and neighboring macroblock pairs are different,a motion vector of a block on the left side of an upper currentmacroblock to be coded is determined using motion vectors of BL3 andBL4. By using such a processing method, even if motion compensation isperformed for neighboring macroblock on a size basis using a differentsize from a macroblock, a process in the direct mode can be performed inconsideration of the difference of the frame structure and the fieldstructure.

Moreover, when motion compensation is performed for neighboringmacroblock on a size basis different from a macroblock, by calculatingan average value of motion vectors of a block included in themacroblock, the calculated value may be a motion vector of themacroblock. Even if motion compensation is performed for neighboringmacroblocks on a size basis using a different size from a macroblock, aprocess in the direct mode can be performed in consideration of thedifference of a frame structure and a field structure.

By the way, as mentioned above, a motion vector is derived, and interpicture prediction coding is performed based on the derived motionvector. As a result, the motion vector derived in the motion vectordetecting unit 108 and the coded predictive error picture are stored ina bit stream on macroblock basis. However, as for a motion vector of amacroblock coded in the direct mode, it is merely described that it iscoded in the direct mode, and the motion vector and reference indicesare not described in a bit stream. FIG. 50 is an illustration showing anexample of a data configuration of a bit stream 700 generated by a bitstream generating unit 104. As shown FIG. 50, in the bit stream 700generated by the bit stream generating unit 104, a Header is providedfor every Picture. The Header contains, for example, items such as anitem RPSL showing the change of a reference frame list 10, and an itemshowing a picture type of the picture and not shown in this figure, andwhen an assignment method of the first reference index 12 and the secondreference index 13 in the frame list 10 is changed form initialsettings, an assignment method after change is described in the itemRPSL.

On the other hand, coded predictive error is recorded on macroblockbasis. For example, when a macroblock is coded using a spatialprediction in the direct mode, a motion vector of the macroblock is notdescribed in an item Block1, and information showing a coding mode isthe direct mode is described in an item PredType. Here, the item Block1is an item in which a predictive error corresponding to the macroblockis described, and the item PredType shows a coding mode of themacroblock. When it is selected which one of the frame structure or thefield structure is used for coding from the viewpoint of theabove-mentioned coding efficiency, information showing the choicebetween the frame structure and the field structure is described.Subsequently, coded predictive error is described in an item CodedRes.When another macroblock is a macroblock coded in an inter pictureprediction coding mode, it is described in the item PredType in an itemBlock2 that a coding mode for the macroblock is the inter predictioncoding mode. Here, the item CodeRes shows a coding mode described andthe item PredType is an item in which a predictive error correspondingto the macroblock is described. In this case, the first reference index12 of the macroblock is further described in an item Ridx0, and thesecond reference index 13 is further described in an item Ridx1 otherthan the coding mode. Reference indices in a block are represented byvariable length code words, and the shorter code length is assigned tothe smaller value. Subsequently, a motion vector of the macroblockduring forward frame reference is described in an item MV0, and a motionvector during backward frame reference is described in an item MV1. Thencoded predictive error is described in the item CodeRes.

FIG. 51 is a block diagram showing a configuration of a moving picturedecoding apparatus 800 which decodes the bit stream 700 shown in FIG.50. The moving picture decoding apparatus 800 is a moving picturedecoding apparatus which decodes the bit stream 700 in which apredictive error including a macroblock coded in the direct mode isdescribed, and includes a bit stream analyzing unit 701, a predictivedifference decoding unit 702, a mode decoding unit 703, a motioncompensation decoding unit 705, a motion vector storing unit 706, aframe memory 707, an add operating unit 708, switches 709 and 710, and amotion vector decoding unit 711. The bit stream analyzing unit 701extracts various data from inputted bit stream 700. Here, various dataincludes information such as information on a coding mode andinformation on a motion vector or the like. Extracted information on acoding mode is outputted to the mode decoding unit 703. On the otherhand, extracted information on a motion vector is outputted to themotion vector decoding unit 705. Furthermore, extracted predictivedifference coding data is outputted to the predictive differencedecoding unit 702. The predictive difference decoding unit 702 decodesinputted predictive difference coding data and generates a predictivedifference picture. Generated predictive difference picture is outputtedto the switch 709. For example, when the switch 709 is connected to aterminal b, a predictive difference picture is outputted to the addoperating unit 708.

The mode decoding unit 703 controls the switches 709 and 710 referringto a coding mode information extracted form a bit stream. When a codingmode is intra picture coding mode, the mode decoding unit 703 controlsto connect the switch 709 with a terminal a and to connect the switch710 with a terminal c. Moreover, the mode decoding unit 703 outputs acoding mode information to the motion compensation decoding unit 705 andthe motion vector decoding unit 711. The motion vector decoding unit 711decodes a coded motion vector inputted from the bit stream analyzingunit 701. Decoded reference picture number and a decoded motion vectorare stored in the motion vector storing unit 706 and outputted to themotion vector compensation decoding unit 705 at the same time.

When a coding mode is the direct mode, the mode decoding unit 703controls to connect the switch 709 with the terminal b and to connectthe switch 710 with a terminal d. Moreover, the mode decoding unit 703outputs a coding mode information to the motion compensation decodingunit 705 and the motion vector decoding unit 711. The motion vectordecoding unit 711 determines a motion vector to be used in the directmode using a motion vector of neighboring macroblock pair and areference picture number stored in the motion vector storing unit 706,when a coding mode is the direct mode. Since the method for determininga motion vector is same as the contents explained for the operation ofthe mode selecting unit 109 shown in FIG. 40, the explanation will beomitted here.

Based on decoded reference picture number and decoded motion vector, themotion compensation decoding unit 705 obtains a motion compensationpicture on macroblock basis from the frame memory 707. The obtainedmotion compensation picture is outputted to the add operating unit 708.The frame memory 707 is a memory storing decoding pictures on framebasis. The add operating unit 708 adds inputted predictive differencepicture to a motion compensation picture, and generates a decodedpicture. The generated decoded picture is outputted to the frame memory707.

As mentioned above, according to this embodiment, even if a neighboringmacroblock pair coded by a frame structure and a neighboring macroblockpair coded by a field structure are mixed in coded neighboringmacroblock pairs corresponding to a current macroblock pair to be codedinclude in a spatial prediction method in the direct mode, a motionvector can be easily calculated.

Note that in the above embodiment, the case that each picture isprocessed on macroblock pair (connecting two macroblocks vertically)basis using either a frame structure or a field structure is explained,however, a process may be performed by switching a frame structure and afield structure on a different basis, for example, on macroblock basis.

Moreover, in above embodiment, the case that a macroblock in a B pictureis processed in the direct mode, however, a macroblock in a P picturecan be processed likewise. When coding and decoding a P picture, eachblock performs motion compensation from only one picture, and areference frame list is only one. Therefore, in order to perform theprocess same as this embodiment in a P picture, the process calculatingtwo motion vectors of a current block to be coded/decoded (the firstreference frame and the second reference frame) in this embodiment maybe changed to a process calculating one motion vector.

In addition, in the above embodiment, the case that predictivegeneration of a motion vector used in the direct mode is performed usingmotion vectors of three neighboring macroblock pairs is explained,however, the number of neighboring macroblock pairs to be used may bedifferent. For example, a case using only a motion vector of aneighboring macroblock pair located on the left side is conceivable.

Twelfth Embodiment

Through storing a program to realize a configuration of the picturecoding method and the picture decoding method mentioned in the aboveembodiment on a storage medium such as a flexible disk, it becomespossible to easily perform the process mentioned in the above embodimenton an independent computer system.

FIG. 52 is an illustration explaining a storage medium which stores aprogram to realize the picture coding method and the decoding method ofthe above first embodiment through eleventh embodiment on a computersystem.

FIG. 52B shows an external view of the flexible disk viewed from thefront, a configuration of a cross section and the flexible disk, andFIG. 52A shows an example of a physical format of a flexible disk as abody of storage medium. A flexible disk FD is contained in a case F, andplural tracks Tr are formed concentrically on the surface of the diskfrom outer to inner radius, and each track is divided into 16 sectors Sein angular direction. Therefore, as for the flexible disk storing theabove-mentioned program, a picture coding method and a picture decodingmethod as the above program are stored in an allocated area on theabove-mentioned flexible disk FD.

FIG. 52C shows a configuration for recording and reading theabove-mentioned program on and from a flexible disk FD. Whenabove-mentioned program is stored on the flexible disk FD, the picturecoding method and a picture decoding method as above-mentioned programare written via a flexible disk drive from a computer system Cs. Whenthe above coding or decoding method is constructed in the computersystem by the program on the flexible disk, the program is read from theflexible disk and transferred to the computer system.

Note that in the above explanation, a flexible disk is used as a storagemedium, however it is possible to perform likewise using an opticaldisk. Moreover, a storage medium is not limited to a flexible disk andmedia capable of storing a program such as a CD-ROM, a memory card and aROM cassette can execute likewise.

Applications of the picture coding method and the picture decodingmethod shown in the above embodiment and a system using the applicationswill be further explained.

FIG. 53 is a block diagram showing an overall configuration of a contentsupply system ex100 for realizing content distribution service. The areafor providing communication service is divided into cells of desiredsize, and cell sites ex107˜ex110 which are fixed wireless stations areplaced in each cell.

In this content supply system ex100, for example, the Internet ex101 isconnected to devices such as a computer ex111, a PDA (Personal DigitalAssistant) ex112, a camera ex113, a cell phone ex114 and a cell phonewith a camera ex115 via the Internet service provider ex102, a telephonenetwork ex104 and cell sites ex107˜ex110.

However, the content supply system ex100 is not limited to theconfiguration as shown in FIG. 53, and may be connected to a combinationof any of them. Also, each device may be connected directly to thetelephone network ex104, not through the cell sites ex107˜ex110.

The camera ex113 is a device such as a digital video camera capable ofshooting moving pictures. The cell phone may be a cell phone of a PDC(Personal Digital Communication) system, a CDMA (Code Division MultipleAccess) system, a W-CDMA (Wideband-Code Division Multiple Access) systemor a GSM (Global System for Mobile Communications) system, a PHS(Personal Handyphone system) or the like.

A streaming server ex103 is connected to the camera ex113 via the cellsite ex109 and the telephone network ex104, and live distribution or thelike using the camera ex113 based on the coded data transmitted from theuser becomes possible. Either the camera ex113 or the server fortransmitting the data may code the data. Also, the moving picture datashot by a camera ex116 may be transmitted to the streaming server ex103via the computer ex111. The camera ex116 is a device such as a digitalcamera capable of shooting still and moving pictures. Either the cameraex116 or the computer ex111 may code the moving picture data. An LSIex117 included in the computer ex111 or the camera ex116 performs codingprocessing. Software for coding and decoding pictures may be integratedinto any type of storage medium (such as a CD-ROM, a flexible disk and ahard disk) that is a recording medium which is readable by the computerex111 or the like. Furthermore, a cell phone with a camera ex115 maytransmit the moving picture data. This moving picture data is the datacoded by the LSI included in the cell phone ex115.

The content supply system ex100 codes contents (such as a music livevideo) shot by users using the camera ex113, the camera ex116 or thelike in the same manner as the above embodiment and transmits them tothe streaming server ex103, while the streaming server ex103 makesstream distribution of the content data to the clients at their request.The clients include the computer ex111, the PDA ex112, the camera ex113,the cell phone ex114 and so on capable of decoding the above-mentionedcoded data. In the content supply system ex100, the clients can thusreceive and reproduce the coded data, and further can receive, decodeand reproduce the data in real time so as to realize personalbroadcasting.

When each device in this system performs coding and decoding, thepicture coding apparatus or the picture decoding apparatus shown in theabove embodiment may be used. A cell phone is explained as an example.

FIG. 54 is an illustration showing the cell phone ex115 using thepicture coding method and the picture decoding method explained in theabove embodiments. The cell phone ex115 has an antenna ex201 forcommunicating with the cell site ex110 via radio waves, a camera unitex203 such as a CCD camera capable of shooting moving and stillpictures, a display unit ex202 such as a liquid crystal display fordisplaying the data obtained by decoding pictures and the like shot bythe camera unit ex203 and received by the antenna ex201, a body unitincluding a set of operation keys ex204, a voice output unit ex208 suchas a speaker for outputting voices, a voice input unit 205 such as amicrophone for inputting voices, a storage medium ex207 for storingcoded or decoded data such as data of moving or still pictures shot bythe camera, data of received e-mails and data of moving or stillpictures, and a slot unit ex206 for attaching the storage medium ex207to the cell phone ex115. The storage medium ex207 stores in itself aflash memory element, a kind of EEPROM (Electrically Erasable andProgrammable Read Only Memory) that is an electrically erasable andrewritable nonvolatile memory, in a plastic case such as a SD card.

Next, the cell phone ex115 is explained with reference to FIG. 55. Inthe cell phone ex115, a main control unit ex311 for overall controllingeach unit of the body unit including the display unit ex202 and theoperation keys ex204 is connected to a power supply circuit unit ex310,an operation input control unit ex304, a picture coding unit ex312, acamera interface unit ex303, a LCD (Liquid Crystal Display) control unitex302, a picture decoding unit ex309, a multiplexing/separating unitex308, a recording and reading unit ex307, a modem circuit unit ex306and a voice processing unit ex305 to each other via a synchronous busex313.

When a call-end key or a power key is turned ON by a user's operation,the power supply circuit unit ex310 supplies respective components withpower from a battery pack so as to activate the digital cell phone witha camera ex115 for making it into a ready state.

In the cell phone ex115, the voice processing unit ex305 converts thevoice signals received by the voice input unit ex205 in conversationmode into digital voice data under the control of the main control unitex311 including a CPU, ROM and RAM, the modem circuit unit ex306performs spread spectrum processing of the digital voice data, and thecommunication circuit unit ex301 performs digital-to-analog conversionand frequency transform of the data, so as to transmit it via theantenna ex201. Also, in the cell phone ex115, the communication circuitunit ex301 amplifies the data received by the antenna ex201 inconversation mode and performs frequency transform and analog-to-digitalconversion for the data, the modem circuit unit ex306 performs inversespread spectrum processing of the data, and the voice processing unitex305 converts it into analog voice data, so as to output it via thevoice output unit 208.

Furthermore, when transmitting an e-mail in data communication mode, thetext data of the e-mail inputted by operating the operation keys ex204on the body unit is sent out to the main control unit ex311 via theoperation input control unit ex304. In the main control unit ex311,after the modem circuit unit ex306 performs spread spectrum processingof the text data and the communication circuit unit ex301 performsdigital-to-analog conversion and frequency transform for it, the data istransmitted to the cell site ex110 via the antenna ex201.

When picture data is transmitted in data communication mode, the picturedata shot by the camera unit ex203 is supplied to the picture codingunit ex312 via the camera interface unit ex303. When it is nottransmitted, it is also possible to display the picture data shot by thecamera unit ex203 directly on the display unit 202 via the camerainterface unit ex303 and the LCD control unit ex302.

The picture coding unit ex312, which includes the picture codingapparatus as explained in the present invention, compresses and codesthe picture data supplied from the camera unit ex203 by the codingmethod used for the picture coding apparatus as shown in theabove-mentioned embodiment so as to transform it into coded picturedata, and sends it out to the multiplexing/separating unit ex308. Atthis time, the cell phone ex115 sends out the voices received by thevoice input unit ex205 during shooting by the camera unit ex203 to themultiplexing/separating unit ex308 as digital voice data via the voiceprocessing unit ex305.

The multiplexing/separating unit ex308 multiplexes the coded picturedata supplied from the picture coding unit ex312 and the voice datasupplied from the voice processing unit ex305 by a predetermined method,the modem circuit unit ex306 performs spread spectrum processing of themultiplexed data obtained as a result of the multiplexing, and thecommunication circuit unit ex301 performs digital-to-analog conversionand frequency transform of the data for transmitting via the antennaex201.

As for receiving data of a moving picture file which is linked to a Webpage or the like in data communication mode, the modem circuit unitex306 performs inverse spread spectrum processing of the data receivedfrom the cell site ex110 via the antenna ex201, and sends out themultiplexed data obtained as a result of the processing to themultiplexing/separating unit ex308.

In order to decode the multiplexed data received via the antenna ex201,the multiplexing/separating unit ex308 separates the multiplexed datainto a bit stream of picture data and a bit stream of voice data, andsupplies the coded picture data to the picture decoding unit ex309 andthe voice data to the voice processing unit ex305 respectively via thesynchronous bus ex313. Next, the picture decoding unit ex309, whichincludes the picture decoding apparatus as explained in the presentinvention, decodes the bit stream of picture data by the decoding methodcorresponding to the coding method as shown in the above-mentionedembodiment to generate reproduced moving picture data, and supplies thisdata to the display unit ex202 via the LCD control unit ex302, and thusmoving picture data included in a moving picture file linked to a Webpage, for instance, is displayed. At the same time, the voice processingunit ex305 converts the voice data into analog voice data, and suppliesthis data to the voice output unit ex208, and thus voice data includedin a moving picture file linked to a Web page, for instance, isreproduced.

The present invention is not limited to the above-mentioned system, andat least either the picture coding apparatus or the picture decodingapparatus in the above-mentioned embodiments can be incorporated into adigital broadcasting system as shown in FIG. 56. Such satellite orterrestrial digital broadcasting has been in the news lately. Morespecifically, a bit stream of video information is transmitted from abroadcast station ex409 to or communicated with a broadcast satelliteex410 via radio waves. Upon receipt of it, the broadcast satellite ex410transmits radio waves for broadcasting, a home-use antenna ex406 with asatellite broadcast reception function receives the radio waves, and atelevision (receiver) ex401 or a set top box (STB) ex407 decodes the bitstream for reproduction. The picture decoding apparatus as shown in theabove-mentioned embodiment can be implemented in the reproducingapparatus ex403 for reading off and decoding the bit stream recorded ona storage medium ex402 that is a storage medium such as a CD and a DVD.In this case, the reproduced video signals are displayed on a monitorex404. It is also conceived to implement the picture decoding apparatusin the set top box ex407 connected to a cable ex405 for a cabletelevision or the antenna ex406 for satellite and/or terrestrialbroadcasting so as to reproduce them on a monitor ex408 of thetelevision ex401. The picture decoding apparatus may be incorporatedinto the television, not in the set top box. Or, a car ex412 having anantenna ex411 can receive signals from the satellite ex410 or the cellsite ex107 for reproducing moving pictures on a display apparatus suchas a car navigation system ex413.

Furthermore, the picture coding apparatus as shown in theabove-mentioned embodiment can code picture signals for recording on astorage medium. As a concrete example, there is a recorder ex420 such asa DVD recorder for recording picture signals on a DVD disk ex421 and adisk recorder for recording them on a hard disk. They can be recorded ona SD card ex422. If the recorder ex420 includes the picture decodingapparatus as shown in the above-mentioned embodiment, the picturesignals recorded on the DVD disk ex421 or the SD card ex422 can bereproduced for display on the monitor ex408.

As the structure of the car navigation system ex413, the structurewithout the camera unit ex203, the camera interface unit ex303 and thepicture coding unit ex312, out of the components shown in FIG. 55, isconceivable. The same goes for the computer ex111, the television(receiver) ex401 and others.

In addition, three types of implementations can be conceived for aterminal such as the above-mentioned cell phone ex114; asending/receiving terminal implemented with both an encoder and adecoder, a sending terminal implemented with an encoder only, and areceiving terminal implemented with a decoder only.

As described above, it is possible to use the moving picture codingmethod or the moving picture decoding method in the above-mentionedembodiments in any of the above-mentioned device and system, and usingthis method, the effects described in the above embodiments can beobtained.

From the invention thus described, it will be obvious that theembodiments of the invention may be varied in many ways. Such variationsare not to be regarded as a departure from the spirit and scope of theinvention, and all such modifications as would be obvious to one skilledin the art are intended for inclusion within the scope of the followingclaims.

INDUSTRIAL APPLICABILITY

The picture coding apparatus according to the present invention isuseful as a picture coding apparatus embedded in a personal computerwith communication facility, a PDA, a broadcast station for digitalbroadcasting and a cell phone or the like.

The picture decoding apparatus according to the present invention isuseful as a picture decoding apparatus equipped for a personal computerwith communication facility, a PDA, a STB for receiving digitalbroadcasting and a cell phone or the like.

The invention claimed is:
 1. A decoding method for decoding a bit streamcorresponding to coded data of a decoding target block included in adecoding target picture in direct mode, wherein the bit stream isgenerated by the following steps: a step of specifying a co-locatedblock which is a block included in a second picture that is differentfrom the coding target picture, the co-located block being located inthe second picture at the same position that the coding target block islocated in the coding target picture; a step of determining a firstmotion vector and a second motion vector of the coding target block forperforming motion compensation on the coding target block, using a thirdmotion vector which is a motion vector of the co-located block; a stepof generating a first predictive image of the coding target block usingthe first motion vector of the coding target block and a secondpredictive image of the coding target block using the second motionvector of the coding target block; a step of generating a predictiveimage of the coding target block based on the first predictive image andthe second predictive image; a step of generating a difference image ofthe coding target block between the coding target block and thepredictive image of the coding target block; and a step of coding thedifference image of the coding target block to obtain coded data of thecoding target block, the decoding method comprising the steps of:specifying a co-located block which is a block included in a secondpicture that is different from the decoding target picture, theco-located block being located in the second picture at the sameposition that the decoding target block is located in the decodingtarget picture; determining a first motion vector and a second motionvector of the decoding target block for performing motion compensationon the decoding target block, using a third motion vector which is amotion vector of the co-located block; generating a first predictiveimage of the decoding target block using the first motion vector of thedecoding target block and a second predictive image of the decodingtarget block using the second motion vector of the decoding targetblock; generating a predictive image of the decoding target block basedon the first predictive image and the second predictive image; decodingthe coded data of the decoding target block to obtain a difference imageof the decoding target block; and reconstructing the decoding targetblock by adding the difference image of the decoding target block andthe predictive image of the decoding target block, wherein, theco-located block is motion-compensated using a first motion vectorcorresponding to a first reference picture of the co-located block and asecond motion vector corresponding to a second reference picture of theco-located block, and wherein, in the case where the first referencepicture of the co-located block is stored in a long-term picture bufferand a second reference picture of the co-located block is stored in ashort-term picture buffer, (i) the third motion vector is determined tobe equal to the first motion vector of the co-located block, and (ii)the first motion vector of the decoding target block is determined to beequal to the third motion vector, and (iii) the second motion vector ofthe decoding target block is determined to be a value of 0, and wherein,in the case where the first reference picture of the co-located block isstored in a short-term picture buffer and the second reference pictureof the co-located block is stored in a long-term picture buffer, (i) thethird motion vector is determined to be equal to the first motion vectorof the co-located block, and (ii) the first and second motion vectors ofthe decoding target block are calculated by scaling the third motionvector based on a difference between display order information of thefirst reference picture of the decoding target block, display orderinformation of the second reference picture of the decoding targetblock, and display order information of the decoding target pictureincluding the decoding target block.
 2. The decoding method according toclaim 1, wherein the first reference picture of the co-located block isa reference picture selected from a first reference picture list inwhich lower identification numbers are assigned with priority to areference picture located prior to the decoding target picture indisplay order, and the second reference picture of the co-located blockis a reference picture selected from a second reference picture list inwhich lower identification numbers are assigned with priority to areference picture located posterior to the decoding target picture indisplay order.
 3. A decoding apparatus for decoding a bit streamcorresponding to coded data of a decoding target block included in adecoding target picture in direct mode, wherein the bit stream isgenerated by using the following units: a unit operable to specify aco-located block which is a block included in a second picture that isdifferent from the coding target picture, the co-located block beinglocated in the second picture at the same position that the codingtarget block is located in the coding target picture; a unit operable todetermine a first motion vector and a second motion vector of the codingtarget block for performing motion compensation on the coding targetblock, using a third motion vector which is a motion vector of theco-located block; a unit operable to generate a first predictive imageof the coding target block using the first motion vector of the codingtarget block and a second predictive image of the coding target blockusing the second motion vector of the coding target block; a unitoperable to generate a predictive image of the coding target block basedon the first predictive image and the second predictive image; a unitoperable to generate a difference image of the coding target blockbetween the coding target block and the predictive image of the codingtarget block; and a unit operable to code the difference image of thecoding target block to obtain coded data of the coding target block, thedecoding apparatus comprising: a specifying unit operable to specify aco-located block which is a block included in a second picture that isdifferent from the decoding target picture, the co-located block beinglocated in the second picture at the same position that the decodingtarget block is located in the decoding target picture; a motion vectordetermining unit operable to determine a first motion vector and asecond motion vector of the decoding target block for performing motioncompensation on the decoding target block, using a third motion vectorwhich is a motion vector of the co-located block; a first and secondpredictive image generating unit operable to generate a first predictiveimage of the decoding target block using the first motion vector of thedecoding target block and a second predictive image of the decodingtarget block using the second motion vector of the decoding targetblock; a predictive image generating unit operable to generate apredictive image of the decoding target block based on the firstpredictive image and the second predictive image; a difference imagedecoding unit operable to decode the coded data of the decoding targetblock to obtain a difference image of the decoding target block; and ablock image reconstructing unit operable to reconstruct the decodingtarget block by adding the difference image of the decoding target blockand the predictive image of the decoding target block, wherein, theco-located block is motion-compensated using a first motion vectorcorresponding to a first reference picture of the collocated block and asecond motion vector corresponding to a second reference picture of theco-located block, and wherein, in the case where the first referencepicture of the co-located block is stored in a long-term picture bufferand a second reference picture of the co-located block is stored in ashort-term picture buffer, (i) the third motion vector is determined tobe equal to the first motion vector of the co-located block, and (ii)the first motion vector of the decoding target block is determined to beequal to the third motion vector, and (iii) the second motion vector ofthe decoding target block is determined to be a value of 0, and wherein,in the case where the first reference picture of the co-located block isstored in a short-term picture buffer and the second reference pictureof the co-located block is stored in a long-term picture buffer, (i) thethird motion vector is determined to be equal to the first motion vectorof the co-located block, and (ii) the first and second motion vectors ofthe decoding target block are calculated by scaling the third motionvector based on a difference between display order information of thefirst reference picture of the decoding target block, display orderinformation of the second reference picture of the decoding targetblock, and display order information of the decoding target pictureincluding the decoding target block.
 4. The decoding apparatus accordingto claim 3, wherein the first reference picture of the co-located blockis a reference picture selected from a first reference picture list inwhich lower identification numbers are assigned with priority to areference picture located prior to the decoding target picture indisplay order, and the second reference picture of the co-located blockis a reference picture selected from a second reference picture list inwhich lower identification numbers are assigned with priority to areference picture located posterior to the decoding target picture indisplay order.